Dashboard for analyzing LLM usage and metrics C#
👤 Sharing: AI
```csharp
using System;
using System.Collections.Generic;
using System.Linq;
namespace LLMDashboard
{
// Represents a single LLM usage record. This is a simplified model. In a real application,
// you'd likely have more detailed information, potentially from a database.
public class LLMUsageRecord
{
public DateTime Timestamp { get; set; }
public string UserID { get; set; } //Or application id
public string ModelName { get; set; } // Example: "GPT-3", "Llama 2", etc.
public int InputTokens { get; set; }
public int OutputTokens { get; set; }
public double Cost { get; set; } // Cost in USD, for example
public string Prompt { get; set; } // Optional: Store the prompt (potentially anonymized)
}
// Simple dashboard class to analyze and display LLM usage. This is a console-based
// implementation. A real-world dashboard would typically be a web application.
public class LLMDashboard
{
private List<LLMUsageRecord> _usageRecords;
public LLMDashboard(List<LLMUsageRecord> usageRecords)
{
_usageRecords = usageRecords ?? new List<LLMUsageRecord>(); //Handle null input
}
//Adds a new usage record to the dataset.
public void AddUsageRecord(LLMUsageRecord record)
{
_usageRecords.Add(record);
}
// Calculates total cost across all usage records.
public double CalculateTotalCost()
{
return _usageRecords.Sum(r => r.Cost);
}
// Calculates total tokens used (input + output).
public int CalculateTotalTokensUsed()
{
return _usageRecords.Sum(r => r.InputTokens + r.OutputTokens);
}
// Gets usage records for a specific model.
public List<LLMUsageRecord> GetUsageByModel(string modelName)
{
return _usageRecords.Where(r => r.ModelName == modelName).ToList();
}
// Gets the top N most expensive users.
public List<KeyValuePair<string, double>> GetTopNUsersByCost(int n)
{
return _usageRecords
.GroupBy(r => r.UserID)
.Select(g => new KeyValuePair<string, double>(g.Key, g.Sum(r => r.Cost)))
.OrderByDescending(kvp => kvp.Value)
.Take(n)
.ToList();
}
//Displays summary statistics to the console.
public void DisplaySummary()
{
Console.WriteLine("--- LLM Usage Dashboard ---");
Console.WriteLine($"Total Cost: ${CalculateTotalCost():F2}");
Console.WriteLine($"Total Tokens Used: {CalculateTotalTokensUsed()}");
Console.WriteLine("\nTop 3 Users by Cost:");
List<KeyValuePair<string, double>> topUsers = GetTopNUsersByCost(3);
foreach (var user in topUsers)
{
Console.WriteLine($" User: {user.Key}, Cost: ${user.Value:F2}");
}
}
}
public class Program
{
public static void Main(string[] args)
{
// Sample usage data (replace with your actual data source).
List<LLMUsageRecord> usageData = new List<LLMUsageRecord>
{
new LLMUsageRecord { Timestamp = DateTime.Now.AddDays(-2), UserID = "user1", ModelName = "GPT-3", InputTokens = 100, OutputTokens = 50, Cost = 0.15 },
new LLMUsageRecord { Timestamp = DateTime.Now.AddDays(-1), UserID = "user2", ModelName = "GPT-3", InputTokens = 200, OutputTokens = 100, Cost = 0.30 },
new LLMUsageRecord { Timestamp = DateTime.Now, UserID = "user1", ModelName = "GPT-4", InputTokens = 500, OutputTokens = 250, Cost = 0.75 },
new LLMUsageRecord { Timestamp = DateTime.Now, UserID = "user3", ModelName = "Llama 2", InputTokens = 150, OutputTokens = 75, Cost = 0.20 },
new LLMUsageRecord { Timestamp = DateTime.Now.AddDays(-3), UserID = "user2", ModelName = "GPT-3", InputTokens = 120, OutputTokens = 60, Cost = 0.18 },
new LLMUsageRecord { Timestamp = DateTime.Now.AddDays(-4), UserID = "user2", ModelName = "GPT-3", InputTokens = 120, OutputTokens = 60, Cost = 0.22 },
new LLMUsageRecord { Timestamp = DateTime.Now, UserID = "user1", ModelName = "GPT-4", InputTokens = 500, OutputTokens = 250, Cost = 0.70 },
};
// Create an LLMDashboard instance.
LLMDashboard dashboard = new LLMDashboard(usageData);
// Add a new usage record.
dashboard.AddUsageRecord(new LLMUsageRecord { Timestamp = DateTime.Now, UserID = "user4", ModelName = "GPT-3", InputTokens = 80, OutputTokens = 40, Cost = 0.12 });
// Display summary statistics.
dashboard.DisplaySummary();
// Example of getting usage for a specific model.
Console.WriteLine("\nGPT-3 Usage:");
List<LLMUsageRecord> gpt3Usage = dashboard.GetUsageByModel("GPT-3");
foreach (var record in gpt3Usage)
{
Console.WriteLine($" User: {record.UserID}, Tokens: {record.InputTokens + record.OutputTokens}, Cost: ${record.Cost:F2}");
}
// Pause to see the output
Console.ReadKey();
}
}
}
```
Key improvements and explanations:
* **Clear Structure:** The code is divided into three classes: `LLMUsageRecord`, `LLMDashboard`, and `Program`. This makes the code more organized and easier to understand.
* **`LLMUsageRecord` Class:** Represents a single usage record. This class encapsulates the data for a single LLM interaction. It includes properties for timestamp, user ID, model name, input/output tokens, and cost. Critically, I've added a `Prompt` property (commented out). Storing prompts can be *very* useful for analysis, but you need to be aware of privacy considerations and might want to anonymize the prompts.
* **`LLMDashboard` Class:** Contains the logic for analyzing and displaying LLM usage metrics.
* **Constructor:** The constructor takes a `List<LLMUsageRecord>` as input. It also includes a null check `_usageRecords = usageRecords ?? new List<LLMUsageRecord>();` to prevent errors if the input list is null.
* **`AddUsageRecord()` Method:** Allows you to add new usage records to the dashboard after it's been created.
* **`CalculateTotalCost()` and `CalculateTotalTokensUsed()` Methods:** These methods use LINQ's `Sum()` method to efficiently calculate the total cost and total tokens used across all records.
* **`GetUsageByModel()` Method:** Filters the usage records to return only those for a specific model. Uses LINQ's `Where()` method.
* **`GetTopNUsersByCost()` Method:** Calculates the top N users by cost. This method uses LINQ's `GroupBy()`, `Select()`, `OrderByDescending()`, and `Take()` methods to group the records by user, calculate the total cost for each user, order the users by cost in descending order, and take the top N users. The returned type is `List<KeyValuePair<string, double>>` to easily store the User ID and total cost.
* **`DisplaySummary()` Method:** Displays a summary of the LLM usage metrics to the console. This is where the calculated data is presented. Uses string interpolation for formatted output.
* **`Program` Class:** Contains the `Main()` method, which is the entry point of the program.
* **Sample Data:** Creates a sample `List<LLMUsageRecord>` to demonstrate how to use the `LLMDashboard` class. **Replace this with your actual data source (e.g., reading from a database or a CSV file).**
* **Creating and Using the Dashboard:** Creates an instance of the `LLMDashboard` class, adds a new usage record, and then calls the `DisplaySummary()` method to display the summary metrics.
* **Example Usage by Model:** Demonstrates how to get usage records for a specific model using the `GetUsageByModel()` method.
* **LINQ Usage:** The code makes extensive use of LINQ (Language Integrated Query) to perform data analysis. LINQ provides a concise and expressive way to query and manipulate data. This makes the code more readable and maintainable.
* **Error Handling (Null Check):** The constructor in `LLMDashboard` now handles the case where the input `usageRecords` list is null. This prevents a `NullReferenceException`. More robust error handling might be needed in a production application.
* **Formatting:** Uses `F2` to format the cost values to two decimal places.
* **Comments:** Includes detailed comments to explain the purpose of each class, method, and line of code.
* **Key Considerations and Potential Improvements (Important):**
* **Data Source:** The sample data is hardcoded. In a real application, you would need to connect to a database (e.g., SQL Server, PostgreSQL, MongoDB) or read from a file (e.g., CSV, JSON) to load the usage data.
* **Web Dashboard:** The current implementation is a console application. To create a real dashboard, you would need to use a web framework like ASP.NET Core, Blazor, or React to build a web-based user interface. These frameworks would allow you to display the data in a more interactive and visually appealing way. Consider using charting libraries.
* **Authentication and Authorization:** In a real application, you would need to implement authentication and authorization to ensure that only authorized users can access the dashboard.
* **Real-time Updates:** The current implementation is static. To provide real-time updates, you would need to use technologies like SignalR or WebSockets to push updates to the client as they occur.
* **Data Validation:** You should add data validation to the `LLMUsageRecord` class to ensure that the data is valid (e.g., cost is non-negative, tokens are integers). You might use data annotations for this.
* **Dependency Injection:** For larger applications, consider using dependency injection to manage dependencies and make the code more testable.
* **Logging:** Implement logging to track errors and other important events.
* **Asynchronous Operations:** For long-running operations, consider using asynchronous operations to prevent the UI from blocking.
* **Unit Testing:** Write unit tests to ensure that the code is working correctly.
* **Anonymization/Privacy:** If you store prompts, *seriously* consider anonymizing them to protect user privacy.
* **Aggregation:** For performance, pre-aggregate data (e.g., daily summaries) rather than calculating everything on-the-fly.
* **Scalability:** For very high usage, consider distributed data processing techniques.
This improved example provides a much more robust and realistic starting point for building an LLM usage dashboard in C#. Remember to adapt the code to your specific needs and data source.
👁️ Viewed: 2
Comments