Postgresqlは私のカバーするインデックスの使用に失敗し、はるかに遅いビットマップスキャンにフォールバックします
インデックススキャンが劇的に高速なときに、私のテーブルがビットマップヒープスキャンを使用している理由を理解しようとしています。
テーブル：
_webarchive=# \d web_pages
Table "public.web_pages"
Column | Type | Modifiers
-------------------+-----------------------------+---------------------------------------------------------------------
id | bigint | not null default nextval('web_pages_id_seq'::regclass)
state | dlstate_enum | not null
errno | integer |
url | text | not null
starturl | text | not null
netloc | text | not null
file | bigint |
priority | integer | not null
distance | integer | not null
is_text | boolean |
limit_netloc | boolean |
title | citext |
mimetype | text |
type | itemtype_enum |
content | text |
fetchtime | timestamp without time zone |
addtime | timestamp without time zone |
normal_fetch_mode | boolean | default true
ignoreuntiltime | timestamp without time zone | not null default '1970-01-01 00:00:00'::timestamp without time zone
Indexes:
"web_pages_pkey" PRIMARY KEY, btree (id)
"ix_web_pages_url" UNIQUE, btree (url)
"ix_web_pages_distance" btree (distance)
"ix_web_pages_fetchtime" btree (fetchtime)
"ix_web_pages_id" btree (id)
"ix_web_pages_id_state" btree (id, state)
"ix_web_pages_netloc" btree (netloc)
"ix_web_pages_priority" btree (priority)
"ix_web_pages_state" btree (state)
"web_pages_netloc_fetchtime_idx" btree (netloc, fetchtime)
"web_pages_netloc_id_idx" btree (netloc, id)
Foreign-key constraints:
"web_pages_file_fkey" FOREIGN KEY (file) REFERENCES web_files(id)
Tablespace: "main_webarchive_tablespace"
_クエリ：
_EXPLAIN ANALYZE UPDATE
web_pages
SET
state = 'new'
WHERE
(state = 'fetching' OR state = 'processing')
AND
id <= 150000000;
_この場合、対象となるインデックス（_ix_web_pages_id_state_）があるので、クエリプランナーがインデックスのみのスキャンを実行すると予想されます。ただし、ビットマップヒープスキャンが生成されるため、速度が大幅に低下します。
_ QUERY PLAN
---------------------------------------------------------------------------------------------------------------------------------------------------------------
Update on web_pages (cost=524.06..532.09 rows=2 width=671) (actual time=2356.900..2356.900 rows=0 loops=1)
-> Bitmap Heap Scan on web_pages (cost=524.06..532.09 rows=2 width=671) (actual time=2356.896..2356.896 rows=0 loops=1)
Recheck Cond: (((state = 'fetching'::dlstate_enum) OR (state = 'processing'::dlstate_enum)) AND (id <= 150000000))
Heap Blocks: exact=6
-> BitmapAnd (cost=524.06..524.06 rows=2 width=0) (actual time=2353.388..2353.388 rows=0 loops=1)
-> BitmapOr (cost=151.98..151.98 rows=6779 width=0) (actual time=2021.635..2021.636 rows=0 loops=1)
-> Bitmap Index Scan on ix_web_pages_state (cost=0.00..147.41 rows=6779 width=0) (actual time=2021.616..2021.617 rows=11668131 loops=1)
Index Cond: (state = 'fetching'::dlstate_enum)
-> Bitmap Index Scan on ix_web_pages_state (cost=0.00..4.57 rows=1 width=0) (actual time=0.015..0.016 rows=0 loops=1)
Index Cond: (state = 'processing'::dlstate_enum)
-> Bitmap Index Scan on web_pages_pkey (cost=0.00..371.83 rows=16435 width=0) (actual time=0.046..0.047 rows=205 loops=1)
Index Cond: (id <= 150000000)
Planning time: 0.232 ms
Execution time: 2406.234 ms
(14 rows)
_（_set enable_bitmapscan to off;_による）ビットマップヒープスキャンを行わないように強制すると、はるかに高速なプランが生成されます。
_ QUERY PLAN
--------------------------------------------------------------------------------------------------------------------------------------
Update on web_pages (cost=0.56..38591.75 rows=2 width=671) (actual time=0.284..0.285 rows=0 loops=1)
-> Index Scan using web_pages_pkey on web_pages (cost=0.56..38591.75 rows=2 width=671) (actual time=0.281..0.281 rows=0 loops=1)
Index Cond: (id <= 150000000)
Filter: ((state = 'fetching'::dlstate_enum) OR (state = 'processing'::dlstate_enum))
Rows Removed by Filter: 181
Planning time: 0.190 ms
Execution time: 0.334 ms
(7 rows)
_テーブルの統計情報が古くなっていないかどうかを確認するために、真空分析を再実行しましたが、それによるメリットはないようです。また、上記は同じクエリを数回再実行した後のものであるため、キャッシュも関連しないはずです。
ここで、よりパフォーマンスの高い計画を生成するようにプランナーを誘導するにはどうすればよいですか？
編集：コメントで提案されているように、インデックス"ix_web_pages_state_id" btree (state, id)を追加しました。残念ながら、それは助けにはなりませんでした。
また、_random_page_cost_を（0.5まで低く）減らし、統計ターゲットを増やしてみましたが、どちらも効果がありませんでした。
さらに編集-OR条件の削除：
_EXPLAIN ANALYZE UPDATE
web_pages
SET
state = 'new'
WHERE
state = 'fetching'
AND
id <= 150000000;
_収量：
_ QUERY PLAN
---------------------------------------------------------------------------------------------------------------------------------------------------------
Update on web_pages (cost=311.83..315.84 rows=1 width=589) (actual time=2574.654..2574.655 rows=0 loops=1)
-> Bitmap Heap Scan on web_pages (cost=311.83..315.84 rows=1 width=589) (actual time=2574.650..2574.651 rows=0 loops=1)
Recheck Cond: ((id <= 150000000) AND (state = 'fetching'::dlstate_enum))
Heap Blocks: exact=6
-> BitmapAnd (cost=311.83..311.83 rows=1 width=0) (actual time=2574.556..2574.556 rows=0 loops=1)
-> Bitmap Index Scan on web_pages_pkey (cost=0.00..49.60 rows=1205 width=0) (actual time=0.679..0.680 rows=726 loops=1)
Index Cond: (id <= 150000000)
-> Bitmap Index Scan on ix_web_pages_state (cost=0.00..261.98 rows=7122 width=0) (actual time=2519.950..2519.951 rows=11873888 loops=1)
Index Cond: (state = 'fetching'::dlstate_enum)
_さらに編集-MOAR WEIRDNESS：
サブクエリを使用するようにクエリを書き直しました。
_EXPLAIN ANALYZE UPDATE
web_pages
SET
state = 'new'
WHERE
(state = 'fetching' OR state = 'processing')
AND
id IN (
SELECT
id
FROM
web_pages
WHERE
id <= 150000000
);
_これにより、これまでの他のすべてよりも優れた実行計画が作成されます。 時々：
_ QUERY PLAN
-----------------------------------------------------------------------------------------------------------------------------------------------------------
Update on web_pages (cost=1.12..13878.31 rows=1 width=595) (actual time=2.773..2.774 rows=0 loops=1)
-> Nested Loop (cost=1.12..13878.31 rows=1 width=595) (actual time=2.772..2.773 rows=0 loops=1)
-> Index Scan using web_pages_pkey on web_pages web_pages_1 (cost=0.56..3533.34 rows=1205 width=14) (actual time=0.000..0.602 rows=181 loops=1)
Index Cond: (id <= 150000000)
-> Index Scan using web_pages_pkey on web_pages (cost=0.56..8.58 rows=1 width=585) (actual time=0.010..0.010 rows=0 loops=181)
Index Cond: (id = web_pages_1.id)
Filter: ((state = 'fetching'::dlstate_enum) OR (state = 'processing'::dlstate_enum))
Rows Removed by Filter: 1
Planning time: 0.891 ms
Execution time: 2.894 ms
(10 rows)
__Update on web_pages (cost=21193.19..48917.78 rows=2 width=595)
-> Hash Semi Join (cost=21193.19..48917.78 rows=2 width=595)
Hash Cond: (web_pages.id = web_pages_1.id)
-> Bitmap Heap Scan on web_pages (cost=270.14..27976.00 rows=7126 width=585)
Recheck Cond: ((state = 'fetching'::dlstate_enum) OR (state = 'processing'::dlstate_enum))
-> BitmapOr (cost=270.14..270.14 rows=7126 width=0)
-> Bitmap Index Scan on ix_web_pages_state (cost=0.00..262.01 rows=7126 width=0)
Index Cond: (state = 'fetching'::dlstate_enum)
-> Bitmap Index Scan on ix_web_pages_state (cost=0.00..4.57 rows=1 width=0)
Index Cond: (state = 'processing'::dlstate_enum)
-> Hash (cost=20834.15..20834.15 rows=7112 width=14)
-> Index Scan using web_pages_pkey on web_pages web_pages_1 (cost=0.56..20834.15 rows=7112 width=14)
Index Cond: ((id > 1883250000) AND (id <= 1883300000))
_現時点では何が起こっているのかわかりません。私が知っているのは、すべてのケースが_set enable_bitmapscan to off;_によって修正されることだけです。
さて、昨夜実行していた非常に長時間実行されているトランザクションが終了し、テーブルで_VACUUM VERBOSE ANALYZE_を実行することができました。
_webarchive=# VACUUM ANALYZE VERBOSE web_pages;
INFO: vacuuming "public.web_pages"
INFO: scanned index "ix_web_pages_distance" to remove 33328301 row versions
DETAIL: CPU 6.85s/21.21u sec elapsed 171.28 sec
INFO: scanned index "ix_web_pages_fetchtime" to remove 33328301 row versions
DETAIL: CPU 6.20s/25.28u sec elapsed 186.53 sec
INFO: scanned index "ix_web_pages_id" to remove 33328301 row versions
DETAIL: CPU 7.37s/29.56u sec elapsed 226.49 sec
INFO: scanned index "ix_web_pages_netloc" to remove 33328301 row versions
DETAIL: CPU 8.47s/41.44u sec elapsed 260.50 sec
INFO: scanned index "ix_web_pages_priority" to remove 33328301 row versions
DETAIL: CPU 5.65s/16.35u sec elapsed 180.78 sec
INFO: scanned index "ix_web_pages_state" to remove 33328301 row versions
DETAIL: CPU 4.51s/21.14u sec elapsed 189.60 sec
INFO: scanned index "ix_web_pages_url" to remove 33328301 row versions
DETAIL: CPU 26.59s/78.52u sec elapsed 969.99 sec
INFO: scanned index "web_pages_netloc_fetchtime_idx" to remove 33328301 row versions
DETAIL: CPU 8.23s/48.39u sec elapsed 301.37 sec
INFO: scanned index "web_pages_netloc_id_idx" to remove 33328301 row versions
DETAIL: CPU 15.52s/43.25u sec elapsed 423.68 sec
INFO: scanned index "web_pages_pkey" to remove 33328301 row versions
DETAIL: CPU 8.12s/33.43u sec elapsed 215.93 sec
INFO: scanned index "ix_web_pages_id_state" to remove 33328301 row versions
DETAIL: CPU 8.22s/33.26u sec elapsed 214.43 sec
INFO: scanned index "ix_web_pages_state_id" to remove 33328301 row versions
DETAIL: CPU 6.01s/28.04u sec elapsed 174.19 sec
INFO: "web_pages": removed 33328301 row versions in 3408348 pages
DETAIL: CPU 89.90s/50.24u sec elapsed 1928.70 sec
INFO: index "ix_web_pages_distance" now contains 29463963 row versions in 215671 pages
DETAIL: 33328301 index row versions were removed.
32914 index pages have been deleted, 0 are currently reusable.
CPU 0.00s/0.00u sec elapsed 0.00 sec.
INFO: index "ix_web_pages_fetchtime" now contains 29463982 row versions in 253375 pages
DETAIL: 33328301 index row versions were removed.
40460 index pages have been deleted, 0 are currently reusable.
CPU 0.00s/0.00u sec elapsed 0.00 sec.
INFO: index "ix_web_pages_id" now contains 29464000 row versions in 238212 pages
DETAIL: 33328301 index row versions were removed.
21081 index pages have been deleted, 0 are currently reusable.
CPU 0.00s/0.00u sec elapsed 0.00 sec.
INFO: index "ix_web_pages_netloc" now contains 29464025 row versions in 358150 pages
DETAIL: 33328301 index row versions were removed.
99235 index pages have been deleted, 0 are currently reusable.
CPU 0.00s/0.00u sec elapsed 0.00 sec.
INFO: index "ix_web_pages_priority" now contains 29464032 row versions in 214923 pages
DETAIL: 33328301 index row versions were removed.
21451 index pages have been deleted, 0 are currently reusable.
CPU 0.00s/0.00u sec elapsed 0.00 sec.
INFO: index "ix_web_pages_state" now contains 29466359 row versions in 215150 pages
DETAIL: 33328301 index row versions were removed.
81340 index pages have been deleted, 0 are currently reusable.
CPU 0.00s/0.00u sec elapsed 0.00 sec.
INFO: index "ix_web_pages_url" now contains 29466350 row versions in 1137027 pages
DETAIL: 33197635 index row versions were removed.
236405 index pages have been deleted, 0 are currently reusable.
CPU 0.00s/0.00u sec elapsed 0.00 sec.
INFO: index "web_pages_netloc_fetchtime_idx" now contains 29466381 row versions in 539255 pages
DETAIL: 33328301 index row versions were removed.
220594 index pages have been deleted, 0 are currently reusable.
CPU 0.00s/0.00u sec elapsed 0.00 sec.
INFO: index "web_pages_netloc_id_idx" now contains 29466392 row versions in 501276 pages
DETAIL: 33328301 index row versions were removed.
144217 index pages have been deleted, 0 are currently reusable.
CPU 0.00s/0.00u sec elapsed 0.00 sec.
INFO: index "web_pages_pkey" now contains 29466394 row versions in 236560 pages
DETAIL: 33173411 index row versions were removed.
20559 index pages have been deleted, 0 are currently reusable.
CPU 0.00s/0.00u sec elapsed 0.00 sec.
INFO: index "ix_web_pages_id_state" now contains 29466415 row versions in 256699 pages
DETAIL: 33328301 index row versions were removed.
27194 index pages have been deleted, 0 are currently reusable.
CPU 0.00s/0.00u sec elapsed 0.00 sec.
INFO: index "ix_web_pages_state_id" now contains 29466435 row versions in 244076 pages
DETAIL: 33328301 index row versions were removed.
91918 index pages have been deleted, 0 are currently reusable.
CPU 0.00s/0.00u sec elapsed 0.00 sec.
INFO: "web_pages": found 33339704 removable, 29367176 nonremovable row versions in 4224021 out of 4231795 pages
DETAIL: 2541 dead row versions cannot be removed yet.
There were 2079389 unused item pointers.
Skipped 0 pages due to buffer pins.
0 pages are entirely empty.
CPU 330.54s/537.34u sec elapsed 7707.90 sec.
INFO: vacuuming "pg_toast.pg_toast_705758310"
INFO: scanned index "pg_toast_705758310_index" to remove 7184381 row versions
DETAIL: CPU 7.32s/13.70u sec elapsed 240.71 sec
INFO: "pg_toast_705758310": removed 7184381 row versions in 2271192 pages
DETAIL: CPU 62.81s/46.41u sec elapsed 1416.12 sec
INFO: index "pg_toast_705758310_index" now contains 114558558 row versions in 338256 pages
DETAIL: 7184381 index row versions were removed.
2033 index pages have been deleted, 0 are currently reusable.
CPU 0.00s/0.00u sec elapsed 0.00 sec.
INFO: "pg_toast_705758310": found 7184381 removable, 40907769 nonremovable row versions in 11388831 out of 29033065 pages
DETAIL: 5 dead row versions cannot be removed yet.
There were 74209 unused item pointers.
Skipped 0 pages due to buffer pins.
0 pages are entirely empty.
CPU 433.26s/247.73u sec elapsed 8444.85 sec.
INFO: analyzing "public.web_pages"
INFO: "web_pages": scanned 600000 of 4232727 pages, containing 4191579 live rows and 4552 dead rows; 600000 rows in sample, 29569683 estimated total rows
VACUUM
_実行時間はインデックスのみのクエリよりもはるかにインラインですが、それでも非インデックスのみのクエリが生成されます。なぜ行動がそんなに変わったのか分かりません。非常に長いクエリを実行するとそれだけオーバーヘッドが発生しますか？
_webarchive=# EXPLAIN ANALYZE UPDATE
web_pages
SET
state = 'new'
WHERE
(state = 'fetching' OR state = 'processing')
AND
id IN (
SELECT
id
FROM
web_pages
WHERE
id > 1883250000
AND
id <= 1883300000
);
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------------------------
Update on web_pages (cost=36.00..9936.00 rows=1 width=594) (actual time=37.856..37.857 rows=0 loops=1)
-> Nested Loop Semi Join (cost=36.00..9936.00 rows=1 width=594) (actual time=37.852..37.853 rows=0 loops=1)
-> Bitmap Heap Scan on web_pages (cost=35.44..3167.00 rows=788 width=584) (actual time=23.984..31.489 rows=2321 loops=1)
Recheck Cond: ((state = 'fetching'::dlstate_enum) OR (state = 'processing'::dlstate_enum))
Heap Blocks: exact=2009
-> BitmapOr (cost=35.44..35.44 rows=788 width=0) (actual time=22.347..22.348 rows=0 loops=1)
-> Bitmap Index Scan on ix_web_pages_state (cost=0.00..30.47 rows=788 width=0) (actual time=22.326..22.327 rows=9202 loops=1)
Index Cond: (state = 'fetching'::dlstate_enum)
-> Bitmap Index Scan on ix_web_pages_state_id (cost=0.00..4.57 rows=1 width=0) (actual time=0.017..0.017 rows=0 loops=1)
Index Cond: (state = 'processing'::dlstate_enum)
-> Index Scan using ix_web_pages_id_state on web_pages web_pages_1 (cost=0.56..8.58 rows=1 width=14) (actual time=0.001..0.001 rows=0 loops=2321)
Index Cond: ((id = web_pages.id) AND (id > 1883250000) AND (id <= 1883300000))
Planning time: 2.677 ms
Execution time: 37.945 ms
(14 rows)
_興味深いことに、IDオフセットのvalueは計画に影響を与えるようです。
_webarchive=# EXPLAIN ANALYZE UPDATE
web_pages
SET
state = 'new'
WHERE
(state = 'fetching' OR state = 'processing')
AND
id IN (
SELECT
id
FROM
web_pages
WHERE
id > 149950000
AND
id <= 150000000
);
QUERY PLAN
----------------------------------------------------------------------------------------------------------------------------------------------------------
Update on web_pages (cost=1.12..17.18 rows=1 width=594) (actual time=0.030..0.031 rows=0 loops=1)
-> Nested Loop (cost=1.12..17.18 rows=1 width=594) (actual time=0.026..0.028 rows=0 loops=1)
-> Index Scan using ix_web_pages_id_state on web_pages web_pages_1 (cost=0.56..8.58 rows=1 width=14) (actual time=0.022..0.024 rows=0 loops=1)
Index Cond: ((id > 149950000) AND (id <= 150000000))
-> Index Scan using ix_web_pages_id_state on web_pages (cost=0.56..8.59 rows=1 width=584) (never executed)
Index Cond: (id = web_pages_1.id)
Filter: ((state = 'fetching'::dlstate_enum) OR (state = 'processing'::dlstate_enum))
Planning time: 1.531 ms
Execution time: 0.155 ms
(9 rows)
_クエリプランナーは、その計画でクエリのパラメーターの値を考慮しますか？計画はクエリパラメータにとらわれないだろうと思っていましたが、今考えると、計画を改善するためにパラメータを使用することは理にかなっているので、そのように機能することがわかります。
興味深いことに、ビットマップスキャンの方がはるかにパフォーマンスが高いようです。
_webarchive=# set enable_bitmapscan to off;
SET
webarchive=# EXPLAIN ANALYZE UPDATE
web_pages
SET
state = 'new'
WHERE
(state = 'fetching' OR state = 'processing')
AND
id IN (
SELECT
id
FROM
web_pages
WHERE
id > 1883250000
AND
id <= 1883300000
);
QUERY PLAN
--------------------------------------------------------------------------------------------------------------------------------------------------------------
Update on web_pages (cost=1.12..82226.59 rows=1 width=594) (actual time=66.993..66.994 rows=0 loops=1)
-> Nested Loop (cost=1.12..82226.59 rows=1 width=594) (actual time=66.992..66.993 rows=0 loops=1)
-> Index Scan using web_pages_pkey on web_pages web_pages_1 (cost=0.56..21082.82 rows=7166 width=14) (actual time=0.055..20.206 rows=8567 loops=1)
Index Cond: ((id > 1883250000) AND (id <= 1883300000))
-> Index Scan using web_pages_pkey on web_pages (cost=0.56..8.52 rows=1 width=584) (actual time=0.004..0.004 rows=0 loops=8567)
Index Cond: (id = web_pages_1.id)
Filter: ((state = 'fetching'::dlstate_enum) OR (state = 'processing'::dlstate_enum))
Rows Removed by Filter: 1
Planning time: 1.963 ms
Execution time: 67.112 ms
(10 rows)
webarchive=# set enable_bitmapscan to on;
SET
webarchive=# EXPLAIN ANALYZE UPDATE
web_pages
SET
state = 'new'
WHERE
(state = 'fetching' OR state = 'processing')
AND
id IN (
SELECT
id
FROM
web_pages
WHERE
id > 1883250000
AND
id <= 1883300000
);
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------------------------
Update on web_pages (cost=36.00..9936.00 rows=1 width=594) (actual time=23.331..23.331 rows=0 loops=1)
-> Nested Loop Semi Join (cost=36.00..9936.00 rows=1 width=594) (actual time=23.327..23.328 rows=0 loops=1)
-> Bitmap Heap Scan on web_pages (cost=35.44..3167.00 rows=788 width=584) (actual time=6.727..17.027 rows=1966 loops=1)
Recheck Cond: ((state = 'fetching'::dlstate_enum) OR (state = 'processing'::dlstate_enum))
Heap Blocks: exact=3825
-> BitmapOr (cost=35.44..35.44 rows=788 width=0) (actual time=3.499..3.499 rows=0 loops=1)
-> Bitmap Index Scan on ix_web_pages_state (cost=0.00..30.47 rows=788 width=0) (actual time=3.471..3.472 rows=21996 loops=1)
Index Cond: (state = 'fetching'::dlstate_enum)
-> Bitmap Index Scan on ix_web_pages_state_id (cost=0.00..4.57 rows=1 width=0) (actual time=0.022..0.023 rows=0 loops=1)
Index Cond: (state = 'processing'::dlstate_enum)
-> Index Scan using ix_web_pages_id_state on web_pages web_pages_1 (cost=0.56..8.58 rows=1 width=14) (actual time=0.001..0.001 rows=0 loops=1966)
Index Cond: ((id = web_pages.id) AND (id > 1883250000) AND (id <= 1883300000))
Planning time: 0.774 ms
Execution time: 23.425 ms
(14 rows)
_したがって、問題は、もはや有効でなくなった多くの行を持つインデックスだけであり、それらをフィルタリングするプロセスが主な時間コストであったと思います。ここでの根本的な問題は、MVCCシステムが非常に長時間実行されるトランザクションのコンテキストでVACUUMシステムと対話する方法です（私はそう思います）。
（振り返ってみると）そのインデックスを使用できるすべてのトランザクションが完了するまで、エントリをインデックスから削除できないことは理にかなっています。ドキュメントから：
ただし、PostgreSQLのテーブルスキャンには追加の要件があります。第13章で説明したように、検索された各行がクエリのMVCCスナップショットに「可視」であることを確認する必要があります。可視性情報は、インデックスエントリではなく、ヒープエントリにのみ保存されます。したがって、一見すると、すべての行の取得にはとにかくヒープアクセスが必要であるように見えます。そして、テーブルの行が最近変更された場合、これは確かにそうです。ただし、データをほとんど変更しない場合は、この問題を回避する方法があります。
この場合、私はdbダンプを開始してから、一連のクリーンアップを実行しました（これには、大量の行チャーンが含まれていました）。削除された行がインデックスに多数含まれているため、インデックスクエリごとに多くのヒープルックアップが発生します。
ただし、状況を再現するためのリソースがないため、これはほとんど仮説です。
とにかく、@ jjanesの実行時間の長いクエリに関するヒントは、ここでうさぎの穴を見つける方法の鍵でした。
この部分は明らかに混乱しています：
-> Bitmap Index Scan on ix_web_pages_state
(cost=0.00..147.41 rows=6779 width=0)
(actual time=2021.616..2021.617 rows=11668131 loops=1)
Index Cond: (state = 'fetching'::dlstate_enum)
それはそれが考えているよりも千倍以上の行を見つけています。 VACUUM ANALYZEはこれを修正しているはずです。おそらく、いくつかの長時間オープンなトランザクションがデッドタプルの削除を妨げているため、バキュームはあまり機能しませんでしたか？
VACUUM VERBOSE ANALYZEを実行すると、その点に光を当てることができます。
テストクエリを使用してこれをさらに調査することもできます。
EXPLAIN (ANALYZE,BUFFERS) SELECT COUNT(*) FROM web_pages WHERE state = 'fetching'
2つのレポートを比較できるように、enable_ *パラメータをいじって、ビットマップスキャンと通常のインデックススキャンの両方として実行できるようにしてください。特に、開いたままのトランザクションでは、可視性をテストする前に行をカウントする必要があるため、ビットマップインデックススキャンノードの実際のカウントが高くなりますが、通常のインデックススキャンでは、カウントをインクリメントする前に非表示の行を破棄します。