Nobl9 Resource Usage Tracker

This repository contains two versions of a script that fetches resource usage data from the Nobl9 Reports API and appends it to a CSV file for tracking usage over time.

Script Versions

1. Main Version: nobl9_resource_usage_tracker.py

• Use case: Interactive use, development, testing
• Authentication: TOML configuration file with multiple contexts
• Features: Context selection, organization-specific CSV files, full error handling

2. Hardcoded Version: nobl9_resource_usage_tracker_hc.py

• Use case: Automation, CI/CD pipelines, non-interactive environments
• Authentication: Hardcoded credentials at the top of the script
• Features: Simplified for automation, no user interaction required

Features

• Authentication: Uses TOML configuration for seamless authentication and context management
• Resource Tracking: Tracks SLOs, SLO Units, Composite SLO Components, Data Sources, and Users
• Historical Data: Appends data to CSV files for trend analysis
• Flexible: Supports multiple contexts and custom CSV file names
• Custom Instances: Supports custom Nobl9 instances with custom base URLs

Prerequisites

1. Python Dependencies:

   pip install -r requirements_nobl9_tracker.txt

2. Nobl9 Configuration:

   • For main version: Ensure you have a TOML configuration file with your Nobl9 credentials
   • For hardcoded version: Edit the credentials at the top of the script

Usage

Main Version (Interactive)

Basic Usage

python nobl9_resource_usage_tracker.py

Hardcoded Version (Automation)

Setup

1. Edit nobl9_resource_usage_tracker_hc.py
2. Update the configuration section at the top:

   CLIENT_ID = "your-actual-client-id"
   CLIENT_SECRET = "your-actual-client-secret"
   ORGANIZATION_ID = "your-actual-organization-id"

Basic Usage

python nobl9_resource_usage_tracker_hc.py

Custom CSV File Name

python nobl9_resource_usage_tracker_hc.py --csv-file automated_usage.csv

Automatic Organization-Based Filenames

When no CSV file is specified, both scripts automatically create organization-specific files:

# Creates: nobl9_resource_usage_dansepich-2KadyONlKmHJ.csv
python nobl9_resource_usage_tracker.py

# Creates: nobl9_resource_usage_your-org-id.csv
python nobl9_resource_usage_tracker_hc.py

Configuration

Main Version: TOML Configuration

Ensure you have a TOML configuration file with your Nobl9 credentials:

# Default location: ~/.config/nobl9/config.toml
# Or provide custom path when prompted

Example TOML structure:

[contexts]
  [contexts.default]
    clientId = "your-client-id"
    clientSecret = "your-client-secret"
    organization = "your-organization"
  
  [contexts.production]
    clientId = "prod-client-id"
    clientSecret = "prod-client-secret"
    organization = "prod-organization"

Hardcoded Version: Script Configuration

Edit the configuration section at the top of nobl9_resource_usage_tracker_hc.py:

# Nobl9 API Credentials
CLIENT_ID = "your-client-id-here"
CLIENT_SECRET = "your-client-secret-here"
ORGANIZATION_ID = "your-organization-id-here"

# Custom Instance Configuration (optional)
IS_CUSTOM_INSTANCE = False
CUSTOM_BASE_URL = None

# Okta Configuration (for custom instances)
OKTA_ORG_URL = None
OKTA_AUTH_SERVER = None

Security Warning: Never commit hardcoded credentials to version control! Consider using environment variables for production use.

Output

The script will:

1. Display a summary of current resource usage
2. Append data to the specified CSV file with the following columns:

Column	Description
timestamp	ISO timestamp when data was collected
organization	Nobl9 organization name
tier	Nobl9 tier (e.g., "Nobl9 Enterprise")
generated_at	When the report was generated by Nobl9
license_end_date	License expiration date
slos_current	Current number of SLOs
slos_peak	Peak number of SLOs
slo_units_current	Current SLO units usage
slo_units_peak	Peak SLO units usage
composite_slo_current	Current composite SLO components
composite_slo_peak	Peak composite SLO components
data_sources_current	Current data sources
data_sources_peak	Peak data sources
users_current	Current users
users_peak	Peak users

Example Output

Nobl9 Resource Usage Tracker
========================================
Loading configuration...
Using context: default
Authenticating...
Successfully authenticated with Nobl9
Fetching resource usage data...

Resource Usage Summary:
   Tier: Nobl9 Enterprise
   Generated: 2024-01-15T10:30:00Z
   License End: 2024-12-31T23:59:59Z

Usage Details:
   SLOs: 15/20
   SLO Units: 45/50
   Composite SLO Components: 5/10
   Data Sources: 8/12
   Users: 12/15

Appending data to nobl9_resource_usage.csv...
Data appended to nobl9_resource_usage.csv
   Timestamp: 2024-01-15T10:30:15.123456
   Organization: my-org
   Tier: Nobl9 Enterprise

Resource usage tracking completed successfully!

Automation

You can automate either script using cron jobs or CI/CD pipelines:

Recommended: Hardcoded Version for Automation

For automation, use the hardcoded version (nobl9_resource_usage_tracker_hc.py) as it requires no user interaction:

Cron Job Example

# Run daily at 9 AM using hardcoded version
0 9 * * * cd /path/to/scripts && python nobl9_resource_usage_tracker_hc.py

CI/CD Pipeline Example

- name: Track Nobl9 Resource Usage
  run: |
    cd scripts
    python nobl9_resource_usage_tracker_hc.py --csv-file daily_usage.csv

Alternative: Main Version with Context

If you prefer to use the main version for automation, specify a context:

Cron Job with Context

# Run daily at 9 AM using main version with specific context
0 9 * * * cd /path/to/scripts && python nobl9_resource_usage_tracker.py --context production

Environment Variables (Future Enhancement)

For production automation, consider using environment variables:

export NOBL9_CLIENT_ID="your-client-id"
export NOBL9_CLIENT_SECRET="your-client-secret" 
export NOBL9_ORGANIZATION="your-organization"
python nobl9_resource_usage_tracker.py

Note: Environment variable support would need to be added to the script if desired.

API Rate Limits

The script respects Nobl9's API rate limits:

• Access Token: 1 request per 3 seconds per organization
• Reports API: Standard rate limits apply

Error Handling

The script includes comprehensive error handling for:

• Missing dependencies
• Invalid TOML configuration
• Authentication failures
• API rate limit exceeded
• Network timeouts
• Invalid responses

Troubleshooting

Common Issues

Main Version (nobl9_resource_usage_tracker.py)

1. "TOML config file not found"

   • Ensure your TOML config exists at ~/.config/nobl9/config.toml
   • Or provide the full path when prompted

2. "No valid contexts found"

   • Check your TOML file has valid contexts with clientId and clientSecret
   • Ensure the TOML format is correct

3. "Context not found"

   • Check available contexts in your TOML file
   • Use --context to specify the exact context

Hardcoded Version (nobl9_resource_usage_tracker_hc.py)

4. "Please set CLIENT_ID in the configuration section"

   • Edit the script and update the CLIENT_ID variable at the top
   • Replace the placeholder value with your actual client ID

5. "Please set CLIENT_SECRET in the configuration section"

   • Edit the script and update the CLIENT_SECRET variable at the top
   • Replace the placeholder value with your actual client secret

6. "Please set ORGANIZATION_ID in the configuration section"

   • Edit the script and update the ORGANIZATION_ID variable at the top
   • Replace the placeholder value with your actual organization ID

Both Versions

7. "Authentication failed"

   • Verify your clientId and clientSecret are correct
   • Check that your organization ID is valid
   • Ensure your credentials have the necessary permissions

8. "Missing required dependency"

   • Install dependencies: pip install -r requirements_nobl9_tracker.txt

Debug Mode

For additional debugging, you can modify the script to include more verbose logging by adding print statements or using Python's logging module.

Contributing

This script follows the patterns established in your existing codebase:

• Uses TOML configuration for authentication and context management (same as get_annotations.py)
• Follows similar authentication patterns
• Maintains consistent error handling
• Uses pandas for data manipulation when needed
• Supports custom Nobl9 instances

License

This script is part of your internal tooling and follows your existing code patterns and conventions.