Querying non-existent ATA accounts is very slow

[IAliceBobI]

Issue Description

When querying the balance of an ATA (Associated Token Account) that does not exist, the response is very slow.

Reproduction Steps

1. Use a new wallet address that has never held USDC
2. Query the USDC ATA balance for this address
3. Observe significant delay before response

Example Scenario

// Example code (pseudo-code)
const usdcMint = new PublicKey("EPjFWdd5AufqSSqeM2qN1xzybapC8G4wEGGkZwyTDt1v"); // USDC mint
const wallet = new Wallet("new-wallet-address-without-usdc-ata");

// This query is slow when ATA doesn't exist
const balance = await connection.getTokenAccountBalance(
  getAssociatedTokenAddressSync(usdcMint, wallet.publicKey)
);

Expected Behavior

• Should return quickly with a "account not found" error or null balance
• Similar to how mainnet-beta handles non-existent accounts

Actual Behavior

• Significant delay (can take several seconds)
• Makes balance checking for new wallets very slow

Environment

• Surfpool version: [Please fill in]
• OS: [Please fill in]

Additional Context

This affects:

• Balance checking for new users
• Batch balance queries where some accounts don't exist
• Token portfolio displays for new wallets

The issue is particularly noticeable when:

• Querying multiple token balances for a new wallet
• Building portfolio displays where most ATAs don't exist yet

[MicaiahReid]

Interesting, I will work on reproducing and get a fix out if I can repro

[MicaiahReid]

Hi @IAliceBobI, I was not able to reproduce. Do you have any more info for me? The result for non-existant accounts is available nearly instantaneously for me. Are you using the latest version of surfpool? Is this happening when you're batching a lot of calls or something?

[0xzrf]

@IAliceBobI yeah I tried reproducing it too, don't think there's any difference

[IAliceBobI]

Hi @MicaiahReid, @0xzrf,

I'''d like to share our test results that may help reproduce the issue.

Our Use Case

Batch query token balances for 50 addresses (SOL + RAY + USDT + USDC)

Version: surfpool v1.0.0

Old Approach (Slow)

For each address and each token, call getTokenAccountBalance directly

• Problem: When ATA doesn'''t exist, single query takes ~30 seconds
• 50 addresses × 4 tokens = ~30 seconds total

New Approach (Fast)

1. Use getMultipleAccounts to batch check if ATAs exist
2. Only call getTokenAccountBalance for existing ATAs

• Result: ~50ms total (600x faster)

Performance Comparison

Method	Time for 50 Addresses
Old Approach	~30 seconds
New Approach	~50ms

Key Finding

The huge performance improvement (600x) when using getMultipleAccounts as a pre-check clearly demonstrates that getTokenAccountBalance has a performance issue with non-existent accounts.

Suggestion to Reproduce

Test with 50 addresses that have never had ATAs created, compare the performance between the two approaches.

Thank you!

[MicaiahReid]

Thanks for the breakdown, @IAliceBobI. Are you able to share these test cases that you used with me?

[IAliceBobI]

Thanks for looking into this issue! Here's a detailed explanation of how we batch-check ATA existence to avoid slow queries:

Surfpool version: 1.0.0

Our Optimization - Batch Pre-check Strategy:

Problem: When querying token balances for multiple addresses, calling getTokenAccountBalance directly on non-existing ATAs is very slow (~30 seconds timeout per query).

Solution: Use getMultipleAccounts to batch-check which ATAs exist, then only query balances for existing ones.

Implementation Details:

async fn query_token_balances_batch(
    &self,
    requests: Vec<(Pubkey, Pubkey)>, // (address, mint) pairs
) -> Vec<(Pubkey, Pubkey, AmountDetail)> {

    // ===== Step 1: Get Token Programs for all mints (parallel, cached) =====
    let mint_programs: HashMap<Pubkey, Pubkey> = {
        let mut map = HashMap::new();
        let mut futs = Vec::new();

        for (_, mint) in &requests {
            futs.push(async move {
                let program = token::get_token_program_with_cache_client(&self.rpc, mint)
                    .await
                    .unwrap_or(TOKEN_PROGRAM);
                (*mint, program)
            });
        }

        let results = futures::future::join_all(futs).await;
        for (mint, program) in results {
            map.insert(mint, program);
        }
        map
    };

    // ===== Step 2: Calculate all ATA addresses locally (no RPC calls) =====
    let ata_requests: Vec<(Pubkey, Pubkey, Pubkey)> = requests // (address, mint, ata)
        .iter()
        .map(|(address, mint)| {
            let token_program = mint_programs.get(mint).unwrap_or(&TOKEN_PROGRAM);

            // ATA = PDA derived from (address, mint, token_program)
            // This is a LOCAL calculation, very fast (< 1ms)
            let ata = get_associated_token_address_with_program_id_fast(
                address,
                mint,
                token_program
            );

            (*address, *mint, ata)
        })
        .collect();

    // ===== Step 3: Batch-check if ATAs exist using getMultipleAccounts =====
    let ata_addresses: Vec<Pubkey> = ata_requests.iter().map(|(_, _, ata)| *ata).collect();

    let accounts_exist: Vec<bool> = match self.rpc.get_multiple_accounts(&ata_addresses).await {
        Ok(accounts) => {
            // Check if account REALY exists:
            // - Must be Some(account)
            // - Must have lamports > 0 (uninitialized accounts have 0 lamports)
            accounts
                .iter()
                .map(|acc| matches!(acc, Some(account) if account.lamports > 0))
                .collect()
        },
        Err(e) => {
            // Fallback: assume all accounts don't exist to avoid slow queries
            warn!("Batch check failed: {}, assuming all accounts don't exist", e);
            vec![false; ata_addresses.len()]
        },
    };

    // Log results
    let exist_count = accounts_exist.iter().filter(|&&x| x).count();
    info!(
        "Batch pre-check complete: total={}, exist={}, not_exist={}",
        ata_addresses.len(),
        exist_count,
        ata_addresses.len() - exist_count
    );

    // ===== Step 4: Only query balances for EXISTING ATAs =====
    let mut results = Vec::with_capacity(ata_requests.len());

    for ((address, mint, _ata), exists) in ata_requests.iter().zip(accounts_exist.iter()) {
        if !exists {
            // Account doesn't exist, return 0 immediately (FAST, no RPC call)
            results.push((
                *address,
                *mint,
                AmountDetail {
                    address: address.to_string(),
                    token_mint: mint.to_string(),
                    symbol: mint.to_string(),
                    amount: "0".to_string(),
                    decimals: 0,
                    amount_formatted: "0".to_string(),
                },
            ));
        } else {
            // Account exists, query actual balance (FAST, account exists)
            let balance = self.query_token_balance(address, mint).await;
            results.push((*address, *mint, balance));
        }
    }

    results
}

Key Points:

1. ATA Calculation is Local:

let ata = get_associated_token_address_with_program_id_fast(address, mint, token_program);
// This uses PDA derivation: PDA::find(..., &[address, mint])
// No RPC call needed, < 1ms per ATA

2. Batch Existence Check:

let accounts = rpc.get_multiple_accounts(&ata_addresses).await?;
// One RPC call for ALL addresses
// Returns Vec<Option<Account>>

3. Existence Criteria:

matches!(acc, Some(account) if account.lamports > 0)
// Not just Some(account), but lamports must be > 0
// Uninitialized ATAs might exist with 0 lamports

4. Avoiding Slow Queries:

if !exists {
    // Return 0 immediately, NO call to getTokenAccountBalance
    // This avoids the 30-second timeout for non-existing accounts
}

Performance Improvement:

• Old approach: 50 addresses × 4 tokens = ~30 seconds (direct queries)
• New approach: ~50ms (batch pre-check + selective queries)
• Speedup: ~600x faster

This optimization has been working great for us. Just wanted to share our approach in case it's helpful for others!

Let me know if you have any questions or need more details.