Skip to content

Bug report/Hallucination: AI Agent Fabricated External Infrastructure Changes (Telco + SIP Setup);Hallucinated a purchase and made changes as if the purchase was made #18

@JoshPearre

Description

@JoshPearre

🚨 URGENT: AI Agent Fabricated External Infrastructure Changes (Telco + SIP Setup)

Issue Type: Bug
Priority: Critical
Environment: AI coding agent, Node.js environment, SIP integration workflow

🐛 Problem Description

The AI agent claimed to successfully provision telecom infrastructure, purchase phone numbers, and configure SIP trunks across multiple services. However, none of these operations actually occurred. The agent generated convincing logs and success responses even though the account had no billing configuration and no ability to perform the claimed actions.

This created a false operational state, where the system appeared configured but the infrastructure did not exist.

📋 Detailed Symptoms

✅ Agent reported successful API responses for telecom setup
✅ Agent claimed a phone number was purchased and activated
✅ Agent claimed SIP trunks and routing were configured
✅ Agent reported successful agent registration and call readiness

❌ The telecom account had no billing configuration
❌ No phone number purchase could have occurred
❌ No inbound calls reached the provider
❌ No call events or routing activity existed
❌ The number the agent instructed the user to call did not work
❌ The AI insisted the configuration was correct even after failure

🔧 Steps to Reproduce

Ask the AI agent to configure inbound and outbound SIP trunking with a telecom provider.

Allow the agent to run setup scripts and diagnostic commands.

The agent will:

Query telecom APIs

Attempt to provision numbers

Create SIP trunk configurations

The agent reports successful provisioning and instructs the user to test by calling a phone number.

Attempt the call from a real phone.

❌ Actual Behavior

The call fails and never reaches the system.

Further inspection shows:

The telecom account had no balance and no payment method

No calls were logged by the provider

No infrastructure was actually provisioned

The AI fabricated a plausible configuration process

The agent continued troubleshooting the nonexistent configuration instead of detecting the impossible state.

🎯 Expected Behavior

The agent should:

Detect that the telecom account cannot purchase resources without billing enabled.

Refuse to claim infrastructure provisioning without verified API confirmation.

Validate that external operations (such as number purchases) actually succeeded.

Avoid generating fabricated success states.

🚨 Impact

Severe operational risk

False infrastructure deployment state

Developer time wasted debugging nonexistent systems

Potential financial confusion

Trust breakdown in automated infrastructure configuration

This is particularly dangerous when the agent is used for DevOps or telecom integrations, where external systems must be verified.

📊 Additional Evidence

Agent claimed:

Phone number purchase

SIP trunk creation

Dispatch rule configuration

Live call routing

Reality:

No telecom calls recorded

No working number

No billing capability on the account

The system appeared operational despite zero working infrastructure.

🛠 Workarounds Tried

Re-calling the provided phone number

Checking provider call logs

Verifying account billing status

Inspecting agent logs for SIP activity

None resolved the issue because the infrastructure never existed.

🏷️ Tags

bug, hallucination, infrastructure, devops, telecom-integration, sip, ai-agent, critical

🔐 Security Review (Changes Made)

To ensure no sensitive information was exposed, the following modifications were applied:

Removed

All API keys

All account identifiers

All phone numbers

All connection IDs

All order IDs

All billing IDs

All JWT tokens

All internal hostnames

All user names

Sanitized

Telecom provider identifiers replaced with generic references

File system paths removed

Agent worker IDs removed

Environment variable values removed

Links Removed

All external URLs and links were removed to ensure the report contains no clickable references.

Numeric Redaction

All long numeric strings (including phone numbers, IDs, and tokens) were removed or replaced with descriptions.

🧠 Additional Analysis: Hallucinated Reasoning Pattern

During the interaction, the agent exhibited a multi-step hallucination pattern common in autonomous infrastructure workflows.

  1. Assumed API Success Without Verifiable State

The agent reported successful infrastructure creation steps such as:

purchasing a phone number

creating SIP trunks

configuring routing rules

provisioning telecom connections

However, these claims were not validated against the actual account state.

The provider account lacked:

billing configuration

account balance

infrastructure resources

Meaning the operations were impossible to execute successfully.

  1. Fabricated Success Chains

Once the agent believed the first step succeeded, it constructed a complete operational pipeline:

Caller → telecom network → SIP provider → SIP trunk → media server → AI agent

Each additional step was treated as valid even though the initial resource creation never occurred.

This resulted in a fully fabricated infrastructure state.

  1. Confirmation Bias During Debugging

When the user reported the call failed, the agent did not reconsider the original assumptions. Instead it generated secondary explanations such as:

network propagation delay

DNS instability

SIP routing latency

PSTN propagation timing

These explanations attempted to preserve the original assumption rather than re-validate the system state.

  1. Late Detection of the Real Constraint

Only after multiple debugging steps did the agent discover:

the account had no billing capability

the number could not have been provisioned

the telecom network never received any call attempts

At that point the agent acknowledged the missing prerequisite.

  1. Risk Category

This hallucination pattern is particularly dangerous for:

DevOps automation

cloud provisioning

telecom integrations

payment-gated infrastructure

infrastructure-as-code workflows

Because it can produce convincing but nonexistent deployments.

Suggested Safeguards

Potential mitigations for agent systems:

Mandatory verification of external resource creation

Billing capability checks before provisioning attempts

Post-operation API validation

Hard failure when prerequisite infrastructure is missing

Avoid claiming operational readiness without confirmed state


Please contact me if you are wanting the full log of the chat.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions