Logging and Monitoring

Logging and Monitoring are two crucial practices in modern software development and operations (DevOps) that ensure the health, performance, and reliability of applications and systems. While often discussed together, they serve distinct yet complementary purposes.

1. Logging:
Logging is the process of recording events, actions, errors, and other relevant data generated by a system or application. These records, known as logs, provide a historical trail of what happened within the software.

* Purpose of Logging:
* Debugging and Troubleshooting: Identifying the root cause of issues, errors, or unexpected behavior.
* Auditing and Security: Tracking user actions, system changes, and potential security breaches.
* Performance Analysis: Understanding bottlenecks or slow operations over time.
* Understanding User Behavior: Gaining insights into how users interact with the application.
* Compliance: Meeting regulatory requirements by maintaining activity records.

* Log Levels (Severity): Logs are typically categorized by severity to help filter and prioritize information:
* DEBUG: Detailed information for developers, typically only enabled in development environments.
* INFO: General runtime information about the application's progress (e.g., 'User logged in', 'Order processed').
* NOTICE: Normal but significant events.
* WARNING: Potentially harmful situations that don't immediately cause an error but might lead to problems (e.g., 'Deprecated function used').
* ERROR: Runtime errors or unexpected conditions that prevent a specific operation from completing (e.g., 'Database query failed').
* CRITICAL: Severe errors that might cause the application or a major component to become unavailable (e.g., 'System ran out of memory').
* ALERT: Action must be taken immediately (e.g., 'Entire website down').
* EMERGENCY: System is unusable.

* Best Practices for Logging:
* Structured Logging: Using formats like JSON for logs makes them machine-readable and easier to process, search, and analyze.
* Contextual Information: Including relevant data like user IDs, request IDs, file paths, and variable states with log messages.
* Centralized Logging: Aggregating logs from multiple services and servers into a central system (e.g., ELK Stack - Elasticsearch, Logstash, Kibana; Splunk) for easier search and analysis.
* Avoid Sensitive Data: Do not log personal identifiable information (PII), passwords, or other sensitive data directly.
* Asynchronous Logging: To prevent logging from impacting application performance.

2. Monitoring:
Monitoring is the continuous process of observing and tracking the health, performance, and availability of a system or application. It involves collecting metrics, logs, and traces to provide real-time insights into the system's operational state.

* Purpose of Monitoring:
* Proactive Issue Detection: Identifying potential problems before they impact users.
* Performance Optimization: Pinpointing bottlenecks and areas for improvement.
* Capacity Planning: Understanding resource utilization to plan for future growth.
* Alerting: Notifying administrators or on-call teams immediately when predefined thresholds are breached.
* SLA Compliance: Ensuring the system meets agreed-upon service level agreements.

* Key Metrics to Monitor:
* Infrastructure: CPU usage, memory usage, disk I/O, network traffic.
* Application: Request rates, error rates, latency, throughput, response times, active users, database query performance.
* Business: Conversion rates, revenue per user, feature usage.

* Types of Monitoring:
* Infrastructure Monitoring: Focuses on servers, databases, network devices.
* Application Performance Monitoring (APM): Monitors application code execution, database calls, external service calls, and overall transaction tracing.
* User Experience (UX) Monitoring: Real User Monitoring (RUM) tracks actual user interactions, while Synthetic Monitoring simulates user paths.
* Log Monitoring: Analyzing aggregated logs for patterns, errors, and trends.

* Monitoring Tools and Concepts:
* Dashboards: Visual representations of metrics for quick status checks (e.g., Grafana, Kibana).
* Alerting Systems: Configurable rules to trigger notifications (email, SMS, Slack) based on metric thresholds (e.g., Prometheus Alertmanager, PagerDuty).
* Metrics Collection Agents: Software that collects data from systems (e.g., Prometheus Node Exporter, Telegraf, New Relic agents).

Relationship between Logging and Monitoring:
Logs provide granular, event-driven data, which is excellent for detailed debugging and forensic analysis. Monitoring, on the other hand, typically aggregates these events (or other metrics) into quantifiable data points to observe trends, set alerts, and get a high-level overview of system health.

Often, monitoring systems will ingest and analyze logs as one of their data sources. For example, a monitoring system might count the number of 'ERROR' level logs in a given timeframe and trigger an alert if that count exceeds a threshold. They are not interchangeable but are complementary and essential for maintaining robust and observable systems.

```php
<?php

// To run this example, you need to install Monolog via Composer:
// 1. Make sure you have Composer installed: https://getcomposer.org/
// 2. In your project directory, run: composer require monolog/monolog
// 3. Make sure 'vendor/autoload.php' exists and is correctly located.

require_once __DIR__ . '/vendor/autoload.php'; // Load Composer's autoloader

use Monolog\Logger;
use Monolog\Handler\StreamHandler;
use Monolog\Formatter\LineFormatter;

// Define the log file path
$logFile = __DIR__ . '/application.log';

// Create a logger instance with a channel name (e.g., 'MyApp')
$log = new Logger('MyApp');

// --- Create a Formatter ---
// This defines how each log entry will be formatted in the output.
// LineFormatter's constructor: (format, datetimeFormat, allowInlineLineBreaks, ignoreEmptyContextAndExtra)
$formatter = new LineFormatter(
    "[%datetime%] %channel%.%level_name%: %message% %context% %extra%\n",
    "Y-m-d H:i:s",
    true, // allowInlineLineBreaks
    true  // ignoreEmptyContextAndExtra
);

// --- Create a Stream Handler ---
// A handler determines where the logs go (e.g., file, database, syslog).
// StreamHandler writes logs to a file or PHP output stream.
// The second argument is the minimum log level this handler should process.
$streamHandler = new StreamHandler($logFile, Logger::DEBUG);

// Set the formatter for the handler
$streamHandler->setFormatter($formatter);

// Add the handler to the logger. A logger can have multiple handlers.
$log->pushHandler($streamHandler);

// --- Example Logging --- 

// 1. Informational message (INFO level)
// This is for general application flow messages.
$log->info('User logged in successfully.', [
    'user_id' => 123,
    'username' => 'john.doe',
    'ip_address' => '192.168.1.100'
]);

// 2. Warning message (WARNING level)
// For situations that are not errors but indicate potential issues.
try {
    // Simulate a missing configuration file scenario
    if (!file_exists('/app/config/settings.json')) {
        throw new Exception("Required configuration file 'settings.json' not found.");
    }
} catch (Exception $e) {
    $log->warning('Configuration file missing.', [
        'exception' => $e->getMessage(),
        'file_path' => '/app/config/settings.json',
        'action_taken' => 'Using default settings'
    ]);
}

// 3. Error message (ERROR level)
// For runtime errors that prevent an operation from completing.
function connectToDatabase() {
    // Simulate a database connection failure
    $dbConnected = (rand(0, 1) === 1); // 50% chance of failure
    return $dbConnected;
}

if (!connectToDatabase()) {
    $log->error('Failed to connect to the database.', [
        'database_host' => 'localhost',
        'port' => 3306,
        'user' => 'root',
        'error_code' => 1045 // Simulated SQL error code
    ]);
}

// 4. Debug message (DEBUG level)
// Very detailed messages, usually only enabled during development or deep troubleshooting.
$processedData = ['item_id' => 456, 'status' => 'processed', 'duration_ms' => 12.5];
$log->debug('Finished processing an item from the queue.', $processedData);

// 5. Critical message (CRITICAL level)
// For severe errors that might cause the application or a major component to become unavailable.
function checkExternalService() {
    // Simulate a critical external service outage
    return false;
}

if (!checkExternalService()) {
    $log->critical('External Payment Gateway is down!', [
        'service' => 'PaymentGateway',
        'impact' => 'Unable to process new orders'
    ]);
}

echo "Logging complete! Check the file: " . realpath($logFile) . "\n";
echo "A new 'application.log' file has been created or updated in this directory.\n";

?>
```