Утечка памяти: как с ней бороться?

Обсуждение установки и настройки поддерживаемых вебсерверов, а также работы с ними.
Аватара пользователя
con
Сообщения: 84
Зарегистрирован: Вс мар 15, 2020 11:58 am

Утечка памяти: как с ней бороться?

Сообщение con » Вт май 11, 2021 3:25 pm

Сборка NGINX+php-fpm
php 7.2
SWAP 2G
ОЗУ 4G

После добавления на сервер нескольких аккаунтов, перестают работать сайты и начала появляться ошибка

Код: Выделить всё

[alert] 1177#0: fork() failed while spawning "worker process" (12: Cannot allocate memory)
При этом, ПУ работает, SSH тоже.
Перезапуск NGINX / php-fpm не помогает. Спасает только перезагрузка сервера.
В теории, проблема скорее всего решится добавлением ОЗУ. Но хотелось бы по грамотно постараться решить проблему, а не топорно, т.к. пред падением ОЗУ была занята процентов на 60-70.

На различных форумахвычитал, что дело в утечке памяти. В качестве решения предлагают отключить modsecurity. Но в настройках NGINX, modsecurity и так выключен

Код: Выделить всё

ModSecurityEnabled off;
на всякий случай еще добавил перед предыдущим правилом

Код: Выделить всё

modsecurity off;
но не уверен, что это решение.

Как избавиться от данной проблемы? Кто знает?
Centos7

Аватара пользователя
con
Сообщения: 84
Зарегистрирован: Вс мар 15, 2020 11:58 am

Re: Утечка памяти: как с ней бороться?

Сообщение con » Вс май 16, 2021 7:16 am

Еще раз апну тему. Очень нужна ваша помощь. Докупил ОЗУ, но не помогло - все равно вылетают ошибки и сайты становятся недоступными. Вычитал уже все форумы, но все пишут про modsecurity, который и так выключен. Вспомнил, что за пару дней до начала этой проблемы обновил ядро. Вряд ли конечно, но может ли быть в нем дело?
Вот логи, после которых падают сайты (1-я часть)

Код: Выделить всё

2021/05/16 00:15:07 [alert] 3601#0: fork() failed while spawning "worker process" (12: Cannot allocate memory)
2021/05/16 00:15:07 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:15:07 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:15:07 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:15:07 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:15:07 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:15:07 [alert] 3601#0: fork() failed while spawning "worker process" (12: Cannot allocate memory)
2021/05/16 00:15:07 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:15:07 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:15:07 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:15:07 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:15:07 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:15:07 [alert] 3601#0: fork() failed while spawning "worker process" (12: Cannot allocate memory)
2021/05/16 00:15:07 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:15:07 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:15:07 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:15:07 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:15:07 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:15:07 [alert] 3601#0: fork() failed while spawning "worker process" (12: Cannot allocate memory)
2021/05/16 00:15:07 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:15:07 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:15:07 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:15:07 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:15:07 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:15:07 [alert] 3601#0: fork() failed while spawning "cache manager process" (12: Cannot allocate memory)
2021/05/16 00:15:07 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:15:07 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:15:07 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:15:07 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:15:07 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:15:07 [alert] 2614#0: *1546568 open socket #275 left in connection 99
2021/05/16 00:15:07 [alert] 2614#0: aborting
2021/05/16 00:18:46 [alert] 3601#0: fork() failed while spawning "worker process" (12: Cannot allocate memory)
2021/05/16 00:18:46 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:18:46 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:18:46 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:18:46 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:18:46 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:18:46 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:18:46 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:18:46 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:18:46 [alert] 3601#0: fork() failed while spawning "cache manager process" (12: Cannot allocate memory)
2021/05/16 00:18:46 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:18:46 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:18:46 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:18:46 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:18:46 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:18:46 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:18:46 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:18:46 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:18:57 [alert] 3601#0: fork() failed while spawning "worker process" (12: Cannot allocate memory)
2021/05/16 00:18:57 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:18:57 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:18:57 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:18:57 [alert] 3601#0: fork() failed while spawning "worker process" (12: Cannot allocate memory)
2021/05/16 00:18:57 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:18:57 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:18:57 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
Последний раз редактировалось con Вс май 16, 2021 7:19 am, всего редактировалось 1 раз.
Centos7

Аватара пользователя
con
Сообщения: 84
Зарегистрирован: Вс мар 15, 2020 11:58 am

Re: Утечка памяти: как с ней бороться?

Сообщение con » Вс май 16, 2021 7:18 am

2-я часть

Код: Выделить всё


2021/05/16 00:18:57 [alert] 3601#0: fork() failed while spawning "worker process" (12: Cannot allocate memory)
2021/05/16 00:18:57 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:18:57 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:18:57 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:18:57 [alert] 3601#0: fork() failed while spawning "worker process" (12: Cannot allocate memory)
2021/05/16 00:18:57 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:18:57 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:18:57 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:18:57 [alert] 3601#0: fork() failed while spawning "cache manager process" (12: Cannot allocate memory)
2021/05/16 00:18:57 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:18:57 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:18:57 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:19:04 [alert] 3601#0: fork() failed while spawning "worker process" (12: Cannot allocate memory)
2021/05/16 00:19:04 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:19:04 [alert] 3601#0: fork() failed while spawning "worker process" (12: Cannot allocate memory)
2021/05/16 00:19:04 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:19:04 [alert] 3601#0: fork() failed while spawning "worker process" (12: Cannot allocate memory)
2021/05/16 00:19:04 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:19:04 [alert] 3601#0: fork() failed while spawning "cache manager process" (12: Cannot allocate memory)
2021/05/16 00:19:04 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:19:11 [alert] 3601#0: fork() failed while spawning "worker process" (12: Cannot allocate memory)
2021/05/16 00:19:11 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:19:11 [alert] 3601#0: fork() failed while spawning "worker process" (12: Cannot allocate memory)
2021/05/16 00:19:11 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:19:11 [alert] 3601#0: fork() failed while spawning "worker process" (12: Cannot allocate memory)
2021/05/16 00:19:11 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:19:11 [alert] 3601#0: fork() failed while spawning "worker process" (12: Cannot allocate memory)
2021/05/16 00:19:11 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:19:11 [alert] 3601#0: fork() failed while spawning "cache manager process" (12: Cannot allocate memory)
2021/05/16 00:19:11 [alert] 3601#0: sendmsg() failed (9: Bad file descriptor)
2021/05/16 00:19:21 [alert] 3601#0: fork() failed while spawning "worker process" (12: Cannot allocate memory)
2021/05/16 00:19:21 [alert] 3601#0: fork() failed while spawning "worker process" (12: Cannot allocate memory)
2021/05/16 00:19:21 [alert] 3601#0: fork() failed while spawning "worker process" (12: Cannot allocate memory)
2021/05/16 00:19:21 [alert] 3601#0: fork() failed while spawning "worker process" (12: Cannot allocate memory)
2021/05/16 00:19:21 [alert] 3601#0: fork() failed while spawning "cache manager process" (12: Cannot allocate memory)
2021/05/16 00:19:28 [alert] 3601#0: fork() failed while spawning "worker process" (12: Cannot allocate memory)
2021/05/16 00:19:28 [alert] 3601#0: fork() failed while spawning "worker process" (12: Cannot allocate memory)
2021/05/16 00:19:28 [alert] 3601#0: fork() failed while spawning "worker process" (12: Cannot allocate memory)
2021/05/16 00:19:28 [alert] 3601#0: fork() failed while spawning "worker process" (12: Cannot allocate memory)
2021/05/16 00:19:28 [alert] 3601#0: fork() failed while spawning "cache manager process" (12: Cannot allocate memory)
2021/05/16 00:19:34 [alert] 3601#0: fork() failed while spawning "worker process" (12: Cannot allocate memory)
2021/05/16 00:19:34 [alert] 3601#0: fork() failed while spawning "worker process" (12: Cannot allocate memory)
2021/05/16 00:19:34 [alert] 3601#0: fork() failed while spawning "worker process" (12: Cannot allocate memory)
2021/05/16 00:19:34 [alert] 3601#0: fork() failed while spawning "worker process" (12: Cannot allocate memory)
2021/05/16 00:19:34 [alert] 3601#0: fork() failed while spawning "cache manager process" (12: Cannot allocate memory)
2021/05/16 00:19:45 [alert] 3601#0: fork() failed while spawning "worker process" (12: Cannot allocate memory)
2021/05/16 00:19:45 [alert] 3601#0: fork() failed while spawning "worker process" (12: Cannot allocate memory)
2021/05/16 00:19:45 [alert] 3601#0: fork() failed while spawning "worker process" (12: Cannot allocate memory)
2021/05/16 00:19:45 [alert] 3601#0: fork() failed while spawning "worker process" (12: Cannot allocate memory)
2021/05/16 00:19:45 [alert] 3601#0: fork() failed while spawning "cache manager process" (12: Cannot allocate memory)
2021/05/16 00:19:51 [alert] 3601#0: fork() failed while spawning "worker process" (12: Cannot allocate memory)
2021/05/16 00:19:51 [alert] 3601#0: fork() failed while spawning "worker process" (12: Cannot allocate memory)
2021/05/16 00:19:51 [alert] 3601#0: fork() failed while spawning "worker process" (12: Cannot allocate memory)
2021/05/16 00:19:51 [alert] 3601#0: fork() failed while spawning "worker process" (12: Cannot allocate memory)
2021/05/16 00:19:51 [alert] 3601#0: fork() failed while spawning "cache manager process" (12: Cannot allocate memory)
Centos7

Аватара пользователя
sbury
Сообщения: 1344
Зарегистрирован: Вт фев 06, 2018 7:51 am

Re: Утечка памяти: как с ней бороться?

Сообщение sbury » Вс май 16, 2021 7:07 pm

у вас свой отдельный сервер с панелью?
и выводы покажите
uname -r
rpm -qa | grep glibc

Аватара пользователя
con
Сообщения: 84
Зарегистрирован: Вс мар 15, 2020 11:58 am

Re: Утечка памяти: как с ней бороться?

Сообщение con » Вс май 16, 2021 8:53 pm

Здравствуйте, у меня brainycp установлена на VPS.

Код: Выделить всё

root ~ # uname -r
3.10.0-1160.25.1.el7.x86_64
root ~ # rpm -qa | grep glibc
glibc-common-2.17-324.el7_9.x86_64
glibc-devel-2.17-324.el7_9.x86_64
glibc-2.17-324.el7_9.x86_64
glibc-headers-2.17-324.el7_9.x86_64
Centos7

Аватара пользователя
sbury
Сообщения: 1344
Зарегистрирован: Вт фев 06, 2018 7:51 am

Re: Утечка памяти: как с ней бороться?

Сообщение sbury » Пн май 17, 2021 6:03 am

покажите тогда скрин htop
и вывод этих команд
df
yum clean all
sysctl -a | grep swap
systemctl --failed
sysctl -a | grep huge
sysctl -a | grep memory

еще тут в панели
index.php?do=memmanager

покажите скриншот Huge Memory
Последний раз редактировалось sbury Пн май 17, 2021 6:46 am, всего редактировалось 3 раза.

grachevma
Сообщения: 35
Зарегистрирован: Пт май 24, 2019 5:19 pm
Откуда: Spb

Re: Утечка памяти: как с ней бороться?

Сообщение grachevma » Пн май 17, 2021 6:16 am

Аналогичная проблема

Аватара пользователя
sbury
Сообщения: 1344
Зарегистрирован: Вт фев 06, 2018 7:51 am

Re: Утечка памяти: как с ней бороться?

Сообщение sbury » Пн май 17, 2021 6:42 am

grachevma писал(а):
Пн май 17, 2021 6:16 am
Аналогичная проблема
вывод команд и скринов, как в прошлом посте

Аватара пользователя
con
Сообщения: 84
Зарегистрирован: Вс мар 15, 2020 11:58 am

Re: Утечка памяти: как с ней бороться?

Сообщение con » Пн май 17, 2021 6:48 am

можете пожалуйста открыть личку? Скину скриншоты
вывод

Код: Выделить всё

root ~ # df
Filesystem     1K-blocks     Used Available Use% Mounted on
devtmpfs         2947844        0   2947844   0% /dev
tmpfs            2971820        0   2971820   0% /dev/shm
tmpfs            2971820   115384   2856436   4% /run
tmpfs            2971820        0   2971820   0% /sys/fs/cgroup
/dev/vda1       61860632 14384256  47459992  24% /
tmpfs             594368        0    594368   0% /run/user/0
root ~ # yum clean all
Loaded plugins: fastestmirror, priorities, versionlock
Cleaning repos: base brainy-glibc_sb brainy-python3_sb extras myrepo-centos updates
Cleaning up list of fastest mirrors
Other repos take up 38 M of disk space (use --verbose for details)
root ~ # sysctl -a | grep swap
vm.swappiness = 50
root ~ # systemctl --failed
0 loaded units listed. Pass --all to see loaded but inactive units, too.
To show all installed unit files use 'systemctl list-unit-files'.
Centos7

grachevma
Сообщения: 35
Зарегистрирован: Пт май 24, 2019 5:19 pm
Откуда: Spb

Re: Утечка памяти: как с ней бороться?

Сообщение grachevma » Пн май 17, 2021 8:01 am

Код: Выделить всё

# df
Filesystem     1K-blocks     Used Available Use% Mounted on
devtmpfs         3044576        0   3044576   0% /dev
tmpfs            3054636        0   3054636   0% /dev/shm
tmpfs            3054636   320084   2734552  11% /run
tmpfs            3054636        0   3054636   0% /sys/fs/cgroup
/dev/vda1       61795116 25138588  33494488  43% /
tmpfs             610928        0    610928   0% /run/user/0

Код: Выделить всё

# yum clean all
Loaded plugins: fastestmirror, langpacks, priorities, versionlock
Cleaning repos: base brainy-glibc_sb brainy-python3_sb extras myrepo-centos timeweb-zabbix updates

Код: Выделить всё

# sysctl -a | grep swap
vm.swappiness = 50

Код: Выделить всё

# systemctl --failed
0 loaded units listed. Pass --all to see loaded but inactive units, too.
To show all installed unit files use 'systemctl list-unit-files'.

Код: Выделить всё

# sysctl -a | grep huge
vm.hugepages_treat_as_movable = 0
vm.hugetlb_shm_group = 0
vm.nr_hugepages = 20
vm.nr_hugepages_mempolicy = 20
vm.nr_overcommit_hugepages = 0

Код: Выделить всё

# sysctl -a | grep memory
vm.memory_failure_early_kill = 0
vm.memory_failure_recovery = 1
vm.overcommit_memory = 0
Вложения
Безымянный2.png
Безымянный2.png (14.25 КБ) 6027 просмотров
Безымянный.png
Безымянный.png (75.14 КБ) 6027 просмотров

Ответить