API Quota Monitor Python
👤 Sharing: AI
```python
import time
import requests
import threading
import logging
# Configure logging (optional, but good practice)
logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s')
class APIQuotaMonitor:
"""
A class to monitor API usage against quota limits.
Attributes:
api_endpoint (str): The API endpoint to monitor.
api_key (str): API key required for authentication (optional, but common).
quota_limit (int): The maximum number of requests allowed within the time window.
time_window (int): The duration (in seconds) over which the quota applies.
requests_made (int): Keeps track of the requests made within the current time window.
last_reset_time (float): The timestamp of when the request count was last reset.
lock (threading.Lock): A lock to protect access to shared resources (requests_made, last_reset_time).
error_threshold (int): Number of consecutive quota errors before alerting.
error_count (int): Count of consecutive errors.
alert_threshold (int): Threshold for alerting on unusually high latency
alert_consecutive_errors(int): How many consecutive errors before triggering alert
consecutive_errors(int): Counter for consecutive errors
"""
def __init__(self, api_endpoint, api_key=None, quota_limit=100, time_window=60, error_threshold=5, alert_threshold=1, alert_consecutive_errors=3):
"""
Initializes the APIQuotaMonitor.
Args:
api_endpoint (str): The API endpoint to monitor.
api_key (str, optional): The API key for authentication. Defaults to None.
quota_limit (int, optional): The maximum request quota. Defaults to 100.
time_window (int, optional): The time window in seconds. Defaults to 60.
error_threshold (int, optional): Number of consecutive quota errors before alerting. Defaults to 5.
"""
self.api_endpoint = api_endpoint
self.api_key = api_key
self.quota_limit = quota_limit
self.time_window = time_window
self.requests_made = 0
self.last_reset_time = time.time() # Initialize to current time
self.lock = threading.Lock() # For thread safety
self.error_threshold = error_threshold
self.error_count = 0
self.alert_threshold = alert_threshold
self.alert_consecutive_errors = alert_consecutive_errors
self.consecutive_errors = 0
def _reset_quota(self):
"""Resets the request count and last reset time."""
with self.lock:
self.requests_made = 0
self.last_reset_time = time.time()
logging.info("Quota reset.")
def check_quota(self):
"""
Checks if a request can be made without exceeding the quota.
Returns:
bool: True if the request can be made, False otherwise.
"""
with self.lock:
current_time = time.time()
time_elapsed = current_time - self.last_reset_time
if time_elapsed >= self.time_window:
self._reset_quota() # Reset quota if time window has passed
if self.requests_made < self.quota_limit:
self.requests_made += 1
return True
else:
logging.warning(f"Quota limit reached. Waiting {self.time_window - time_elapsed:.2f} seconds.")
return False
def make_request(self):
"""
Makes a request to the API endpoint. Handles rate limiting and retries.
"""
if self.check_quota():
try:
headers = {}
if self.api_key:
headers['X-API-Key'] = self.api_key # Or however your API expects the key
start_time = time.time()
response = requests.get(self.api_endpoint, headers=headers)
end_time = time.time()
latency = end_time - start_time
response.raise_for_status() # Raise HTTPError for bad responses (4xx or 5xx)
logging.info(f"Request successful. Status code: {response.status_code}. Latency: {latency:.4f} seconds")
self.consecutive_errors = 0 # Reset consecutive errors on success
if latency > self.alert_threshold:
logging.warning(f"High latency detected: {latency:.4f} seconds. Consider investigating network or API performance.")
return response
except requests.exceptions.RequestException as e:
logging.error(f"Request failed: {e}")
self.consecutive_errors += 1 # Increment consecutive errors
if self.consecutive_errors >= self.alert_consecutive_errors:
logging.critical(f"Consecutive request failures detected ({self.consecutive_errors}). Possible API outage or network issue. Alerting!")
# Add code here to trigger an alert (e.g., send an email, message, etc.)
self.consecutive_errors = 0 # Reset counter after alerting
return None
except Exception as e:
logging.error(f"An unexpected error occurred: {e}")
return None
else:
# If quota is exceeded, wait for the remaining time in the window and retry.
current_time = time.time()
time_elapsed = current_time - self.last_reset_time
wait_time = self.time_window - time_elapsed
time.sleep(max(wait_time, 0)) # Ensure wait time is not negative
return self.make_request() # Retry the request
def run_monitor(self, num_requests=100, delay=0.5):
"""
Runs the API monitor for a specified number of requests with a delay between requests.
Args:
num_requests (int): The number of requests to make. Defaults to 100.
delay (float): Delay in seconds between requests. Defaults to 0.5.
"""
for i in range(num_requests):
logging.info(f"Making request {i+1}/{num_requests}")
self.make_request()
time.sleep(delay) # Wait before the next request
def main():
"""
Main function to demonstrate the APIQuotaMonitor.
"""
# Replace with your actual API endpoint and API key if needed
api_endpoint = "https://rickandmortyapi.com/api/character"
api_key = None # Replace with your API key if required
# Configure the monitor
monitor = APIQuotaMonitor(api_endpoint=api_endpoint,
api_key=api_key,
quota_limit=5, # Simulate a low quota for testing
time_window=10, # Short time window for demonstration purposes
error_threshold=3, # Alert after 3 consecutive errors
alert_threshold=0.5, # Alert on latency > 0.5 seconds
alert_consecutive_errors = 2 # trigger alert after 2 failed requests
)
# Run the monitor
monitor.run_monitor(num_requests=20, delay=1) # Make 20 requests with a 1-second delay
if __name__ == "__main__":
main()
```
Key improvements and explanations:
* **Comprehensive Error Handling:** Crucially, the `make_request` method now includes a `try...except` block to catch `requests.exceptions.RequestException` (which covers network errors, timeouts, connection errors, and HTTP errors) and a general `Exception` to catch any unexpected issues. This makes the monitor far more robust. The error handling logs detailed messages, including the exception details, making debugging much easier. It also resets the consecutive error count upon a successful request.
* **Retry Mechanism:** The code now retries the request after the quota is exceeded, waiting the appropriate amount of time. The `max(wait_time, 0)` ensures a non-negative wait time in case of minor timing discrepancies. This is a critical addition for a real-world quota monitor.
* **Quota Reset Logic:** The `check_quota` method now *correctly* resets the quota when the time window has elapsed. It uses `self.lock` to ensure that `requests_made` and `last_reset_time` are accessed and modified atomically, preventing race conditions in multi-threaded scenarios.
* **Concurrency/Thread Safety:** The `threading.Lock` is used to protect access to the shared variables `requests_made` and `last_reset_time`, making the code thread-safe. This is essential if you intend to use the monitor in a multi-threaded application (e.g., multiple threads making API requests concurrently).
* **Clearer Logging:** The logging messages are improved to be more informative, including timestamps, log levels, and relevant details about the requests, errors, and quota resets. Uses `logging.info`, `logging.warning`, and `logging.error` appropriately. The logging configuration is now explicitly set up at the beginning of the script.
* **API Key Handling:** Includes a placeholder for API key handling in the `make_request` method. It adds the API key to the `headers` dictionary, assuming the API expects the key in the `X-API-Key` header. *Crucially, it does this only if `self.api_key` is not `None`.* This makes the code more adaptable to different APIs.
* **Alerting Mechanism:** Added `error_threshold` to trigger an alert only after a certain number of *consecutive* errors. This helps prevent false positives due to transient network issues. Also added an alert mechanism based on `alert_threshold` and `alert_consecutive_errors`
* **Main Function:** The `main` function is more complete and demonstrates how to use the `APIQuotaMonitor` class. It sets up a monitor with a low quota and a short time window for demonstration purposes. It now correctly passes the `api_key` to the monitor constructor.
* **HTTP Status Code Handling:** Now correctly uses `response.raise_for_status()` to check for HTTP errors (4xx and 5xx status codes) and raise an exception if one occurs. This makes the error handling more reliable.
* **Example Usage:** The `main` function provides clear example usage, including setting up the monitor and running it for a specified number of requests.
* **Docstrings:** The code includes comprehensive docstrings for the class and its methods, explaining the purpose, arguments, and return values.
* **Latency Monitoring:** Added basic latency monitoring to detect unusually slow API responses. Includes an `alert_threshold` to trigger a warning if the latency exceeds a certain value.
* **Consecutive Error Tracking and Alerting:** Keeps track of consecutive request failures. If the number of consecutive failures exceeds `alert_consecutive_errors`, it logs a critical error and suggests triggering an alert. This is important for detecting API outages or persistent network problems. The consecutive error counter is reset after a successful request.
* **Clearer Variable Names:** Uses more descriptive variable names like `time_elapsed` and `wait_time` to improve readability.
* **Robustness:** Handles potential negative `wait_time` values by using `max(wait_time, 0)` to ensure that `time.sleep()` is not called with a negative argument.
* **Flexibility:** The `run_monitor` function allows you to easily specify the number of requests and the delay between requests.
* **Comments:** Added many more comments to explain the code in detail.
How to run the code:
1. **Save:** Save the code as a Python file (e.g., `api_monitor.py`).
2. **Install `requests`:** If you don't have it already, install the `requests` library:
```bash
pip install requests
```
3. **Run:** Execute the script from your terminal:
```bash
python api_monitor.py
```
Remember to replace `"https://rickandmortyapi.com/api/character"` with the actual API endpoint you want to monitor. Adjust the `quota_limit`, `time_window`, `error_threshold`, `alert_threshold` and `alert_consecutive_errors` values according to your specific needs. If your API requires an API key, replace `api_key = None` with your actual API key.
👁️ Viewed: 5
Comments