01 / Performance
High server load and CPU spikes
Investigate a slow or unresponsive website by identifying whether the pressure comes from processes, memory, I/O, traffic or a dependent service.
Symptoms
Slow page responses, elevated load average, resource alerts or intermittent timeouts.
Diagnostic approach
- Review
uptime, top/htop, ps, free -m and iostat if available. - Correlate resource spikes with web access/error logs and database activity.
- Check for unusual request rates, process counts and memory pressure.
Resolution strategy
Address the confirmed cause: tune a service, mitigate abusive traffic, review caching or escalate application/database optimisation with supporting evidence.
Validation
Recheck load, CPU, memory, response times and logs after the change. Confirm that legitimate traffic continues to work.
top / htopLogsCapacity analysisRCA
02 / Database
MySQL/MariaDB high CPU and slow queries
Narrow a database bottleneck to long-running queries, lock contention, excessive connections or resource constraints.
Symptoms
Slow dynamic pages, high database CPU, connection saturation or request timeouts.
Diagnostic approach
- Run
SHOW FULL PROCESSLIST; and review active connections. - Inspect slow-query logs and relevant Performance Schema data.
- Look for waiting queries, locks, repeated expensive queries and connection spikes.
Resolution strategy
Identify the affected query/workload and provide evidence-based recommendations. Make schema or index changes only with authorisation, testing and a rollback plan.
Validation
Compare query duration, active connections, CPU and application response times before and after the change.
MySQLProcess listSlow query log
03 / Web stack
HTTP 502 Bad Gateway and backend recovery
Trace reverse-proxy failures to the correct layer instead of restarting services blindly.
Symptoms
502 responses, intermittent gateway errors or a site failing after a deployment or traffic spike.
Diagnostic approach
- Review Apache/Nginx error logs and proxy/upstream settings.
- Check PHP-FPM or application service health with
systemctl status. - Verify ports/sockets, permissions, resource limits and OOM-killer logs.
Resolution strategy
Correct the verified backend, socket, proxy or resource issue. Restart only when justified and safe; preserve logs for root-cause analysis.
Validation
Test the application path with curl, confirm expected HTTP status codes and review logs for recurring errors.
ApacheNginxPHP-FPMHTTP 502
04 / Security
DDoS-like traffic, bots and WAF false positives
Separate abusive traffic from legitimate requests and avoid breaking customer workflows while improving protection.
Symptoms
Sudden traffic spikes, repeated requests from suspicious sources, 403 errors or contact forms blocked by a WAF.
Diagnostic approach
- Review access logs, request paths, rates and Cloudflare/WAF event details.
- Compare blocked requests with timestamps and application behaviour.
- Check whether the issue is edge filtering, origin load or a legitimate rule match.
Resolution strategy
Apply proportionate rate limits or narrow rule exceptions with authorisation. Avoid broad allowlists or disabling security controls without evidence.
Validation
Confirm abusive patterns are mitigated while legitimate browsing, login and form submission continue to work.
CloudflareWAFRate limitingAccess logs
05 / Access
SSH access failures and firewall review
Restore administrative connectivity without exposing management ports more broadly than necessary.
Symptoms
SSH timeout, connection refused, authentication failure or an administrator locked out after a firewall change.
Diagnostic approach
- Check
systemctl status sshd and ss -lntp. - Inspect authentication and system logs; verify the configured SSH port.
- Review the active firewall manager and source-address restrictions.
Resolution strategy
Fix the actual service/configuration/rule issue and permit only intended administrator sources where possible. Keep console access or a rollback path available.
Validation
Test a fresh SSH connection, confirm authentication logs and verify the firewall remains restrictive.
SSHsystemdCSF / firewall
06 / TLS
SSL/TLS certificate and HTTPS errors
Identify expiry, hostname, chain or virtual-host issues that prevent clients from establishing trusted HTTPS connections.
Symptoms
Browser certificate warnings, TLS handshake failures or an unexpected certificate being served.
Diagnostic approach
- Check hostname, validity dates, SAN entries and full chain.
- Use
openssl s_client to inspect the certificate served by the endpoint. - Verify virtual-host/SNI configuration and certificate file paths.
Resolution strategy
Install the correct certificate and intermediate chain, fix the matching virtual host or renew the certificate through the approved process.
Validation
Retest HTTPS externally, verify the served certificate and review web-server logs.
SSL/TLSOpenSSLSNI
07 / Networking
DNS resolution and propagation issues
Determine whether a domain issue comes from incorrect records, nameserver delegation, resolver caching or normal TTL behaviour.
Symptoms
Domain resolves to the wrong IP, website or email records fail, or resolution differs between networks.
Diagnostic approach
- Compare authoritative and recursive responses with
dig. - Check A/AAAA, CNAME, MX, NS and relevant TXT records.
- Inspect delegation, TTLs and recent zone changes.
Resolution strategy
Correct the record or delegation at the authoritative source, then allow for applicable TTL/cache behaviour rather than promising instant propagation.
Validation
Query authoritative nameservers and multiple resolvers; verify the website and mail destinations resolve as expected.
DNSdigMX / TXT
08 / Storage
Disk-full and inode exhaustion incidents
Find what is consuming storage and restore capacity without deleting data blindly.
Symptoms
Failed writes, database errors, mail queue growth, service failures or alerts for 100% filesystem usage.
Diagnostic approach
- Check
df -h and df -i. - Use
du to identify large directories and review log/temporary-file growth. - Check for deleted files still held open by processes when usage does not reconcile.
Resolution strategy
Use approved log rotation, safe cleanup or storage expansion. Preserve required logs and database files; do not remove unknown production data.
Validation
Confirm free blocks and inodes, verify affected services and investigate why growth occurred.
dfduInodesLog rotation
09 / Email
Email delivery and mailbox troubleshooting
Trace delivery failures using SMTP responses, mail queues, server logs and domain authentication records.
Symptoms
Delayed or bounced messages, mail in spam, SMTP rejection or mailbox access problems.
Diagnostic approach
- Review Exim/Postfix logs and queue state.
- Interpret SMTP response codes and verify sender/recipient routing.
- Check MX, SPF, DKIM, DMARC, SMTP connectivity and Dovecot health.
Resolution strategy
Correct the confirmed routing, authentication, queue, service or reputation issue; coordinate with the receiving provider when rejection is remote.
Validation
Send a controlled test, confirm delivery logs and verify authentication alignment where applicable.
Exim / PostfixDovecotSPF / DKIM / DMARC
10 / Incident response
Suspicious activity and malware triage
Collect evidence, contain confirmed threats and support safe recovery from a potential website or server compromise.
Symptoms
Unexpected processes, modified files, phishing content, suspicious scheduled tasks or unexplained resource usage.
Diagnostic approach
- Review process lists, authentication logs, file changes, cron entries and scanner findings.
- Correlate evidence with web logs and known deployment activity.
- Preserve relevant logs and document the scope before remediation.
Resolution strategy
Follow the incident process to contain the threat, remove confirmed malicious content, patch the entry point and rotate exposed credentials as appropriate.
Validation
Rescan, verify service behaviour, review access controls and document preventive actions.
Log analysisContainmentHardening
11 / Reliability
Website migration, backup and recovery planning
Reduce migration risk with a repeatable checklist, data validation, testing and rollback preparation.
Risks
Missing files, database mismatch, incorrect permissions, broken configuration, DNS issues or extended downtime.
Preparation
- Inventory services, versions, dependencies, DNS and scheduled jobs.
- Verify a restorable backup and plan data synchronisation.
- Prepare destination tests and a rollback decision point.
Execution strategy
Transfer files and databases securely, validate configuration and permissions, test before cutover and coordinate DNS changes.
Validation
Test critical URLs, forms, database operations, mail and TLS; retain the rollback path until acceptance checks pass.
MigrationBackup / restoreRollback
12 / Operations
Root cause analysis and recurring issue prevention
Turn a one-time recovery into a documented and repeatable support outcome.
Evidence collection
Build a timeline from alerts, logs, metrics, recent changes and service dependencies. Separate confirmed facts from assumptions.
Root cause
Explain the initiating fault and contributing conditions, rather than documenting only the visible symptom.
Corrective action
Record the fix, approvals, risk, rollback method and validation checks. Coordinate cross-team tasks with clear ownership.
Prevention
Recommend actionable monitoring, alert thresholds, runbooks, capacity reviews, patching or configuration changes where appropriate.
RCAL2 / L3 escalationRunbooksPrevention