サーバー上のMySQL最適化
約80〜100のアクティブなWebサイトを実行しているアクティブなサーバーがあります。サーバーを最適化し、RAMの量を減らし、MySQL設定を最適化します。30GBのRAMが割り当てられ、12コア、CentOS7 。私はMySQLTunerを実行して、次の推奨事項を得ました。
MySQLチューナーの推奨事項：
MySQLTuner 1.7.19 - Major Hayden <[email protected]>
[--] Skipped version check for MySQLTuner script
[!!] Your MySQL version 5.5.66-MariaDB is EOL software! Upgrade soon!
[OK] Operating on 64-bit architecture
-------- Performance Metrics -----
[--] Up for: 31d 2h 31m 13s (309M q [115.168 qps], 1M conn, TX: 2455G, RX: 159G
[--] Reads / Writes: 95% / 5%
[--] Binary logging is disabled
[--] Physical Memory : 29.3G
[--] Max MySQL memory : 24.6G
[--] Other process memory: 0B
[--] Total buffers: 13.7G global + 22.4M per thread (500 max threads)
[--] P_S Max memory usage: 0B
[--] Galera GCache Max memory usage: 0B
[OK] Maximum reached memory usage: 14.5G (49.49% of installed RAM)
[OK] Maximum possible memory usage: 24.6G (84.00% of installed RAM)
[OK] Overall possible memory usage with other process is compatible with memory available
[OK] Slow queries: 0% (96/309M)
[OK] Highest usage of available connections: 7% (38/500)
[OK] Aborted connections: 0.59% (7936/1345660)
[!!] name resolution is active : a reverse name resolution is made for each new connection and can reduce performance
[!!] Query cache may be disabled by default due to mutex contention.
[OK] Query cache efficiency: 47.6% (263M cached / 553M selects)
[!!] Query cache prunes per day: 208303
[OK] Sorts requiring temporary tables: 0% (431 temp sorts / 2M sorts)
[!!] Joins performed without indexes: 37167
[OK] Temporary tables created on disk: 6% (31K on disk / 457K total)
[OK] Thread cache hit rate: 99% (38 created / 1M connections)
[!!] Table cache hit rate: 0% (400 open / 1M opened)
[!!] table_definition_cache(400) is lower than number of tables(34357)
[OK] Open file limit used: 0% (4/2K)
[OK] Table locks acquired immediately: 99% (45M immediate / 45M locks)
-------- Performance schema ---------
[--] Performance schema is disabled.
[--] Memory used by P_S: 0B
[--] Sys schema isn't installed.
-------- ThreadPool Metrics --------
[--] ThreadPool stat is enabled.
[--] Thread Pool Size: 12 thread(s).
[--] Using default value is good enough for your version (5.5.66-MariaDB)
-------- MyISAM Metrics -----------
[!!] Key buffer used: 18.7% (588M used / 3B cache)
[OK] Key buffer size / total MyISAM indexes: 2.9G/750.1M
[!!] Read Key buffer hit rate: 7.2% (78K cached / 72K reads)
[!!] Write Key buffer hit rate: 2.1% (2K cached / 60 writes)
-------- InnoDB Metrics ----------
[--] InnoDB is enabled.
[--] InnoDB Thread Concurrency: 0
[OK] InnoDB File per table is activated
[OK] InnoDB buffer pool / data size: 10.0G/6.1G
[OK] Ratio InnoDB log file size / InnoDB Buffer pool size: 1.0G * 2/10.0G should be equal to 25%
[!!] InnoDB buffer pool instances: 1
[--] InnoDB Buffer Pool Chunk Size not used or defined in your version
[OK] InnoDB Read buffer efficiency: 99.99% (2352937489 hits/ 2353198127 total
[OK] InnoDB Write log efficiency: 96.40% (264231435 hits/ 274111596 total)
[OK] InnoDB log waits: 0.00% (0 waits / 9880161 writes)
-------- AriaDB Metrics ----------
[--] AriaDB is enabled.
[OK] Aria pagecache size / total Aria indexes: 128.0M/1B
[OK] Aria pagecache hit rate: 100.0% (5M cached / 2K reads)
-------- TokuDB Metrics ----------
[--] TokuDB is disabled.
-------- XtraDB Metrics ---------
[--] XtraDB is disabled.
-------- Galera Metrics --------
[--] Galera is disabled.
-------- Replication Metrics -----
[--] Galera Synchronous replication: NO
[--] No replication slave(s) for this server.
[--] Binlog format: STATEMENT
[--] XA support enabled: ON
[--] Semi synchronous replication Master: Not Activated
[--] Semi synchronous replication Slave: Not Activated
[--] This is a standalone server
-------- Recommendations ----------
General recommendations:
Control warning line(s) into /var/lib/mysql/localhost.localdomain.err file
Control error line(s) into /var/lib/mysql/localhost.localdomain.err file
Increasing the query_cache size over 128M may reduce performance
We will suggest raising the 'join_buffer_size' until JOINs not using indexes are found.
See https://dev.mysql.com/doc/internals/en/join-buffer-size.html
(specially the conclusions at the bottom of the page).
Increase table_open_cache gradually to avoid file descriptor limits
Read this before increasing table_open_cache over 64: https://bit.ly/1mi7c4C
Read this before increasing for MariaDB
https://mariadb.com/kb/en/library/optimizing-table_open_cache/
This is MyISAM only table_cache scalability problem, InnoDB not affected.
See more details here: https://bugs.mysql.com/bug.php?id=49177
This bug already fixed in MySQL 5.7.9 and newer MySQL versions.
Beware that open_files_limit (2505) variable
should be greater than table_open_cache (400)
Variables to adjust:
SET innodb_stats_on_metadata = OFF
query_cache_size (=0)
query_cache_type (=0)
query_cache_size (> 500M) [see warning above]
join_buffer_size (> 128.0K, or always use indexes with JOINs)
table_open_cache (> 400)
table_definition_cache(400) > 34357 or -1 (autosizing if supported)
innodb_buffer_pool_instances(=10)
my.cnfおよびその他のcnfs
[mysqld]
innodb_file_per_table
slow_query_log
long_query_time=5
slow_query_log_file=/opt/db-backups1/mysqllog/mysql_slow.log
[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0
#######################################################################
#innodb_force_recovery=4
#######################################################################
max_connections = 500
thread_cache_size=2000
max_allowed_packet = 1024M
key_buffer_size = 3000M
read_buffer_size = 16M
read_rnd_buffer_size = 4M
bulk_insert_buffer_size = 256M
myisam_sort_buffer_size = 16M
myisam_max_sort_file_size = 256M
myisam_repair_threads = 1
innodb_buffer_pool_size = 10G
innodb_log_file_size = 1G
innodb_log_buffer_size = 100M
query_cache_limit = 60M
query_cache_size = 500M
query_cache_type = 1
推奨事項をありがとう、彼らは大歓迎です。一部のMyISAMデータベースを古いサーバーから物理的に移行し、一部の空のテーブルには対応するテーブルファイルがなく、システムがそれを欠落しているとシステムが判断したため、エラーログが表示されます。 MyISAMデータベースを物理的にコピーする代わりに、適切なツールを使用してエクスポートおよびインポートすることが、その解決策になると思います。アクセス率が指数関数的に上昇する日付と月があり、サーバーがこの重い負荷に備える準備ができていることに言及することが重要です。
編集1：トップとulimitの結果を追加する
[root@www ~]# cat topres.txt
top - 16:55:44 up 34 days, 3:59, 1 user, load average: 0,02, 0,08, 0,12
Tasks: 204 total, 3 running, 201 sleeping, 0 stopped, 0 zombie
%Cpu(s): 4,6 us, 1,5 sy, 0,0 ni, 93,9 id, 0,0 wa, 0,0 hi, 0,0 si, 0,0 st
KiB Mem : 30714928 total, 3197812 free, 13266944 used, 14250172 buff/cache
KiB Swap: 8257532 total, 7476220 free, 781312 used. 16947012 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
14463 Apache 20 0 674920 42676 19516 R 43,8 0,1 0:00.34 httpd
2965 mysql 20 0 16,3g 12,0g 7744 S 18,8 41,0 1117:52 mysqld
14496 root 20 0 162048 2296 1540 R 6,2 0,0 0:00.01 top
1 root 20 0 190980 3228 2032 S 0,0 0,0 4:07.76 systemd
2 root 20 0 0 0 0 S 0,0 0,0 0:00.91 kthreadd
ulimit -a：
[root@www ~]# ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 119888
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 119888
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
編集2：サーバー上の高トラフィック後にmysqlチューナーの結果を追加します。どんな推薦でも大歓迎です。プロセッサの量を16の仮想CPUに変更しました。
>> MySQLTuner 1.7.19 - Major Hayden <[email protected]>
>> Bug reports, feature requests, and downloads at http://mysqltuner.com/
>> Run with '--help' for additional options and output filtering
[--] Skipped version check for MySQLTuner script
[!!] Your MySQL version 5.5.66-MariaDB is EOL software! Upgrade soon!
[OK] Operating on 64-bit architecture
-------- Log file Recommendations ---------------------------------------------- --------------------
[OK] Log file /var/lib/mysql/localhost.localdomain.err exists
[--] Log file: /var/lib/mysql/localhost.localdomain.err(29M)
[OK] Log file /var/lib/mysql/localhost.localdomain.err is readable.
[OK] Log file /var/lib/mysql/localhost.localdomain.err is not empty
[OK] Log file /var/lib/mysql/localhost.localdomain.err is smaller than 32 Mb
[!!] /var/lib/mysql/localhost.localdomain.err contains 8 warning(s).
[!!] /var/lib/mysql/localhost.localdomain.err contains 5699 error(s).
[--] 8 start(s) detected in /var/lib/mysql/localhost.localdomain.err
[--] 1) 200220 19:00:40 [Note] /usr/sbin/mysqld: ready for connections.
[--] 2) 200220 18:42:35 [Note] /usr/sbin/mysqld: ready for connections.
[--] 3) 200114 12:55:56 [Note] /usr/sbin/mysqld: ready for connections.
[--] 4) 200114 12:54:30 [Note] /usr/sbin/mysqld: ready for connections.
[--] 5) 200114 9:43:11 [Note] /usr/sbin/mysqld: ready for connections.
[--] 6) 200114 9:39:02 [Note] /usr/sbin/mysqld: ready for connections.
[--] 7) 200110 15:38:03 [Note] /usr/sbin/mysqld: ready for connections.
[--] 8) 200110 15:28:21 [Note] /usr/sbin/mysqld: ready for connections.
[--] 7 shutdown(s) detected in /var/lib/mysql/localhost.localdomain.err
[--] 1) 200220 19:00:35 [Note] /usr/sbin/mysqld: Shutdown complete
[--] 2) 200114 12:55:30 [Note] /usr/sbin/mysqld: Shutdown complete
[--] 3) 200114 12:52:41 [Note] /usr/sbin/mysqld: Shutdown complete
[--] 4) 200114 9:42:12 [Note] /usr/sbin/mysqld: Shutdown complete
[--] 5) 200114 9:38:55 [Note] /usr/sbin/mysqld: Shutdown complete
[--] 6) 200110 15:37:55 [Note] /usr/sbin/mysqld: Shutdown complete
[--] 7) 200110 15:22:13 [Note] /usr/sbin/mysqld: Shutdown complete
-------- Storage Engine Statistics -----------------------------------------------------------------
[--] Status: +ARCHIVE +Aria +BLACKHOLE +CSV +FEDERATED +InnoDB +MEMORY +MRG_MYISAM +MyISAM +PERFORMANCE_SCHEMA
[--] Data in MyISAM tables: 290.0M (Tables: 3)
[--] Data in InnoDB tables: 6.3G (Tables: 33793)
[OK] Total fragmented tables: 0
-------- Analysis Performance Metrics --------------------------------------------------------------
[--] innodb_stats_on_metadata: ON
[!!] Stat are updated during querying INFORMATION_SCHEMA.
-------- Security Recommendations ------------------------------------------------------------------
[OK] There are no anonymous accounts for any database users
[OK] All database users have passwords assigned
-------- CVE Security Recommendations --------------------------------------------------------------
[OK] NO SECURITY CVE FOUND FOR YOUR VERSION
-------- Performance Metrics -----------------------------------------------------------------------
[--] Up for: 10d 13h 51m 8s (89M q [97.936 qps], 313K conn, TX: 627G, RX: 68G)
[--] Reads / Writes: 95% / 5%
[--] Binary logging is disabled
[--] Physical Memory : 29.3G
[--] Max MySQL memory : 24.6G
[--] Other process memory: 0B
[--] Total buffers: 13.7G global + 22.4M per thread (500 max threads)
[--] P_S Max memory usage: 0B
[--] Galera GCache Max memory usage: 0B
[OK] Maximum reached memory usage: 14.0G (47.84% of installed RAM)
[OK] Maximum possible memory usage: 24.6G (84.00% of installed RAM)
[OK] Overall possible memory usage with other process is compatible with memory available
[OK] Slow queries: 0% (1/89M)
[OK] Highest usage of available connections: 3% (16/500)
[OK] Aborted connections: 0.00% (1/313391)
[!!] name resolution is active : a reverse name resolution is made for each new connection and can reduce performance
[!!] Query cache may be disabled by default due to mutex contention.
[OK] Query cache efficiency: 47.7% (76M cached / 161M selects)
[!!] Query cache prunes per day: 172476
[OK] Sorts requiring temporary tables: 0% (40 temp sorts / 658K sorts)
[!!] Joins performed without indexes: 6650
[!!] Temporary tables created on disk: 32% (71K on disk / 220K total)
[OK] Thread cache hit rate: 99% (16 created / 313K connections)
[!!] Table cache hit rate: 0% (400 open / 509K opened)
[!!] table_definition_cache(400) is lower than number of tables(33899)
[OK] Open file limit used: 0% (4/2K)
[OK] Table locks acquired immediately: 99% (12M immediate / 12M locks)
-------- Performance schema ------------------------------------------------------------------------
[--] Performance schema is disabled.
[--] Memory used by P_S: 0B
[--] Sys schema isn't installed.
-------- ThreadPool Metrics ------------------------------------------------------------------------
[--] ThreadPool stat is enabled.
[--] Thread Pool Size: 16 thread(s).
[--] Using default value is good enough for your version (5.5.66-MariaDB)
-------- MyISAM Metrics ----------------------------------------------------------------------------
[!!] Key buffer used: 18.7% (588M used / 3B cache)
[OK] Key buffer size / total MyISAM indexes: 2.9G/370.0K
[!!] Read Key buffer hit rate: 63.9% (366 cached / 132 reads)
-------- InnoDB Metrics ----------------------------------------------------------------------------
[--] InnoDB is enabled.
[--] InnoDB Thread Concurrency: 0
[OK] InnoDB File per table is activated
[OK] InnoDB buffer pool / data size: 10.0G/6.3G
[OK] Ratio InnoDB log file size / InnoDB Buffer pool size: 1.0G * 2/10.0G should be equal to 25%
[!!] InnoDB buffer pool instances: 1
[--] InnoDB Buffer Pool Chunk Size not used or defined in your version
[OK] InnoDB Read buffer efficiency: 99.92% (548727530 hits/ 549178090 total)
[OK] InnoDB Write log efficiency: 96.50% (76869307 hits/ 79658631 total)
[OK] InnoDB log waits: 0.00% (0 waits / 2789324 writes)
-------- AriaDB Metrics ----------------------------------------------------------------------------
[--] AriaDB is enabled.
[OK] Aria pagecache size / total Aria indexes: 128.0M/1B
[OK] Aria pagecache hit rate: 100.0% (295K cached / 124 reads)
-------- TokuDB Metrics ----------------------------------------------------------------------------
[--] TokuDB is disabled.
-------- XtraDB Metrics ----------------------------------------------------------------------------
[--] XtraDB is disabled.
-------- Galera Metrics ----------------------------------------------------------------------------
[--] Galera is disabled.
-------- Replication Metrics -----------------------------------------------------------------------
[--] Galera Synchronous replication: NO
[--] No replication slave(s) for this server.
[--] Binlog format: STATEMENT
[--] XA support enabled: ON
[--] Semi synchronous replication Master: Not Activated
[--] Semi synchronous replication Slave: Not Activated
[--] This is a standalone server
-------- Recommendations ---------------------------------------------------------------------------
General recommendations:
Control warning line(s) into /var/lib/mysql/localhost.localdomain.err file
Control error line(s) into /var/lib/mysql/localhost.localdomain.err file
Configure your accounts with ip or subnets only, then update your configuration with skip-name-resolve=1
Increasing the query_cache size over 128M may reduce performance
We will suggest raising the 'join_buffer_size' until JOINs not using indexes are found.
See https://dev.mysql.com/doc/internals/en/join-buffer-size.html
(specially the conclusions at the bottom of the page).
When making adjustments, make tmp_table_size/max_heap_table_size equal
Reduce your SELECT DISTINCT queries which have no LIMIT clause
Increase table_open_cache gradually to avoid file descriptor limits
Read this before increasing table_open_cache over 64: https://bit.ly/1mi7c4C
Read this before increasing for MariaDB https://mariadb.com/kb/en/library/optimizing-table_open_cache/
This is MyISAM only table_cache scalability problem, InnoDB not affected.
See more details here: https://bugs.mysql.com/bug.php?id=49177
This bug already fixed in MySQL 5.7.9 and newer MySQL versions.
Beware that open_files_limit (2505) variable
should be greater than table_open_cache (400)
Variables to adjust:
SET innodb_stats_on_metadata = OFF
query_cache_size (=0)
query_cache_type (=0)
query_cache_size (> 500M) [see warning above]
join_buffer_size (> 128.0K, or always use indexes with JOINs)
tmp_table_size (> 16M)
max_heap_table_size (> 16M)
table_open_cache (> 400)
table_definition_cache(400) > 33899 or -1 (autosizing if supported)
innodb_buffer_pool_instances(=10)
今月のCPUとメモリの概要。 
1日のCPUとメモリの概要。 
Innodbのチューナーの結果は、6.1Gのデータがあることを示しているため、innodb_buffer_pool_sizeを5Gに減らして、「InnoDB読み取りバッファー効率」にどのような影響があるかを確認できます。
これにより、mysqlの16.3g ram使用量が11.3Gに減少します。現在合計で約13Gを使用していることを考えると、割り当てを約16Gに下げるのはかなり安全です。
「使用可能な接続の最大使用量」は、38での最大使用接続数を示しています。これは、設定した上限500を大幅に下回っています。ラムを落とそうとしているので、最大値を100に下げることをお勧めします。 500接続に十分なメモリがない可能性があります。
クエリキャッシュを無効にする必要があります。その拡張性の問題で、とにかく43％の効率しか得られませんでした。
long_query_time 5は大きすぎます。 5秒のページフェッチはおそらく許容できないため、これを低くします。すべてのチューナーの結果から、スロークエリログには80クエリしかないことがわかります。
遅いクエリを見てください。それらをより速くすることは、利用可能な同じ数の接続でより高い負荷が可能であることを意味します。例えば1秒間のクエリを30ミリ秒で実行すると、1秒あたりのクエリ数が多くなります。
34357テーブル??一体何が起こっているのですか？おそらくWordPressで、顧客ごとに1つのdbがありますか？それともそのようなものですか？とにかく、table_open_cacheを400よりもはるかに大きく設定する必要があります。4000を試してください。table_definition_cacheについても同様です。 OSが各プロセスに許可するものを増やす必要があるかもしれません。 ulimit -nを参照してください。
long_query_timeを1（秒）に下げ、slowlogをオンにします。それはあなたがいくつかのいたずらなクエリをキャッチできるようになります。
115 QPS？それはかなり低いです。どのような問題が発生していますか？
クエリキャッシュの使用は不審です。 query_cache_size = 500Mは、各「プルーン」で多くの時間を費やすことを意味します。毎秒数十のプルーンがあるので、良いことよりも害が大きいことを示唆しています。プランA：5,000万に減らします。プランB：0に落とします（ほとんどの「本番」サーバーでは、QCは良いというよりは害があります）。
thread_cache_size=2000-不必要に大きい。 100にドロップします。
「RAMの量を減らす可能性があります」-なぜですか？現在、すべてを使用しているわけではなく、「大きな」設定（key_buffer_sizeおよびinnodb_buffer_pool_size）が必要以上に大きくなっています。つまり、これらの設定では成長の余地があり、それらを引き上げる余地さえあります。または、より安価なマシンに縮小することを考えているということですか？もしそうなら、私はそれを念頭に置いて数字を調べなければなりません。できれば、以下の分析を行った後。
設定に関する別の見解を得るために http://mysql.rjweb.org/doc.php/mysql_analysis を参照し、さらにスローログを分析してください。