> ## Documentation Index
> Fetch the complete documentation index at: https://docs.phylax.systems/llms.txt
> Use this file to discover all available pages before exploring further.

# Backtesting

> Test assertions against historical blockchain transactions

Backtesting runs your assertions against actual historical transactions from a specified block range. This helps ensure your assertions behave as expected with real transaction patterns before deployment.

<Note>
  If you're looking for the reference documentation on all configuration parameters, see the [Backtesting Reference](/credible/backtesting-reference).
</Note>

<Note>
  Backtesting is a development-time validation tool. Running a backtest does not deploy assertions, change the on-chain registry, or affect production enforcement.
</Note>

## How It Works

Backtesting operates in two phases:

1. **Transaction Fetching**: Identifies all transactions to the target contract in the block range using trace APIs (with automatic fallback)
2. **Transaction Validation**: Replays each transaction with your assertion enabled

The tool automatically detects both direct calls and internal calls (e.g., when Contract A calls your target Contract B through a router or aggregator) using trace APIs.

For validation, the tool forks the EVM at the exact transaction state and replays it against the specified assertion.

## Example

```solidity theme={null}
// SPDX-License-Identifier: MIT
pragma solidity ^0.8.20;

import {CredibleTestWithBacktesting} from "credible-std/CredibleTestWithBacktesting.sol";
import {BacktestingTypes} from "credible-std/utils/BacktestingTypes.sol";
import {MyAssertion} from "../assertions/src/MyAssertion.a.sol";

contract MyBacktestingTest is CredibleTestWithBacktesting {
    function testHistoricalTransactions() public {
        BacktestingTypes.BacktestingResults memory results = executeBacktest(
            BacktestingTypes.BacktestingConfig({
                targetContract: 0x5fd84259d66Cd46123540766Be93DFE6D43130D7, // USDC on Optimism Sepolia
                endBlock: 31336940, // Latest block to test
                blockRange: 20, // Number of blocks to test
                assertionCreationCode: type(MyAssertion).creationCode,
                assertionSelector: MyAssertion.assertionInvariant.selector,
                rpcUrl: "https://sepolia.optimism.io",
                detailedBlocks: false,
                forkByTxHash: true
            })
        );

        // Check results
        assertEq(results.assertionFailures, 0, "Found protocol violations!");
    }
}
```

Backtesting tests inherit from `CredibleTestWithBacktesting` instead of `CredibleTest`. The `executeBacktest` function returns results in a `BacktestingTypes.BacktestingResults` struct.

<Tip>
  Consider excluding backtesting from your CI/CD pipeline as it can take time to run. Run backtesting manually on demand to increase confidence in your assertions before deployment.
</Tip>

## Common Configurations

### Standard Configuration

```solidity theme={null}
BacktestingTypes.BacktestingConfig({
    targetContract: TARGET_CONTRACT,
    endBlock: LATEST_BLOCK,
    blockRange: 100,
    assertionCreationCode: type(MyAssertion).creationCode,
    assertionSelector: MyAssertion.assertionInvariant.selector,
    rpcUrl: vm.envString("MAINNET_RPC_URL"),
    detailedBlocks: false,
    forkByTxHash: true
})
```

### Testing Against a Specific Block

When you want to test your assertion against a specific known transaction (such as a historical exploit), set `blockRange: 1` with the block number containing that transaction:

```solidity theme={null}
BacktestingTypes.BacktestingConfig({
    targetContract: BALANCER_V2_VAULT,
    endBlock: 23717632,     // Block containing the exploit transaction
    blockRange: 1,          // Only test this single block
    assertionCreationCode: type(BatchSwapDeltaAssertion).creationCode,
    assertionSelector: BatchSwapDeltaAssertion.assertionBatchSwapRateManipulation.selector,
    rpcUrl: vm.envString("MAINNET_RPC_URL"),
    detailedBlocks: false,
    forkByTxHash: true
})
```

This is particularly useful for:

* Validating that your assertion catches known exploits
* Testing edge cases found in production
* Verifying assertion behavior on specific problematic transactions

<Tip>
  See the [Balancer V2 Rate Manipulation Exploit](/assertions-book/previous-hacks/balancer-v2-stable-rate-exploit) for a real-world example of using backtesting to verify an assertion catches a historical exploit transaction.
</Tip>

### Testing a Single Transaction by Hash

For debugging a staging finding or validating against a specific transaction, use `executeBacktestForTransaction`:

```solidity theme={null}
bytes32 txHash = 0x1234567890abcdef1234567890abcdef1234567890abcdef1234567890abcdef;

BacktestingTypes.BacktestingResults memory results = executeBacktestForTransaction(
    txHash,
    TARGET_CONTRACT,
    type(MyAssertion).creationCode,
    MyAssertion.assertionInvariant.selector,
    vm.envString("MAINNET_RPC_URL")
);
```

This is particularly useful for:

* Debugging staging findings (see [How to Plan Assertion Testing](/credible/testing-strategy))
* Quick validation of assertion behavior on a specific transaction
* Investigating incidents without needing to know the block number

## Understanding Results

When backtesting completes, transactions are categorized into different result types to help you quickly identify issues:

### Result Categories

**PASS** - Assertion passed successfully

* Transaction replayed and assertion validated without errors
* **Recommended Action:** None - everything is working as expected

**SKIP** - Assertion not triggered

* Transaction called a function not monitored by your assertion (function selector didn't match)
* This is normal behavior - your assertion only triggers on specific function calls
* **Recommended Action:** None required, unless you expected this transaction to trigger your assertion
* **Note:** These are NOT assertion violations - they indicate transactions that aren't relevant to your assertion

**REPLAY\_FAIL** - Transaction replay failed

* Transaction failed to replay before the assertion could execute
* Common causes:
  * **State dependencies:** Transaction depends on specific state that isn't present
  * **Context requirements:** Transaction requires specific block context
  * **Insufficient funds:** Sender balance changed between original execution and replay
* **Recommended Action:** Check the error message for details; this usually indicates an environmental issue rather than an assertion problem
* **Note:** These are NOT assertion violations

**FAIL** - Assertion reverted

* Your assertion reverted when validating this transaction
* **Most Common Causes:**
  * **False positive:** Your assertion logic incorrectly flags legitimate protocol behavior (most common)
  * **Gas limit exceeded:** Assertion ran out of gas (300k limit)
  * **Assertion bug:** Logic error in your assertion code
  * **Known exploit:** If testing against a historical exploit block, this is expected behavior
* **Recommended Action:**
  * Check if the transaction is a known exploit - if so, this confirms your assertion works correctly
  * Review your assertion logic to ensure it properly handles this transaction pattern
  * Check if the assertion is running out of gas and needs optimization
  * Verify the transaction on a block explorer to understand what it does

**ERROR** - Unexpected failure

* An error occurred that doesn't fit other categories
* May indicate RPC issues, assertion bugs, or unexpected contract behavior
* **Recommended Action:** Check the error message and retry; if persistent, [file a bug report](https://github.com/phylaxsystems/credible-std/issues).

### Reading the Summary

At the end of each backtest run, you'll see a summary like:

```text theme={null}
==========================================
           BACKTESTING SUMMARY
==========================================
Block Range: 23697580 - 23697590
Total Transactions: 15
Processed Transactions: 15
Successful Validations: 10
Skipped Transactions: 3
Failed Validations: 2

=== ERROR BREAKDOWN ===
Protocol Violations (Assertion Failures): 1
Replay Failures (Tx reverted before assertion): 1

Success Rate: 83%
================================
```

## Running Tests

```bash theme={null}
# Set RPC URL environment variable
export RPC_URL="YOUR_RPC_URL"

# Run backtesting tests
pcl test --ffi --match-test testHistoricalTransactions
```

<Note>
  The `--ffi` flag is required to enable foreign function interface for RPC calls.
</Note>

## Foundry Configuration

Enable FFI in your Foundry profile:

```toml theme={null}
[profile.backtest-assertions]
src = "assertions/src"
test = "assertions/test/backtest"
out = "assertions/out"
libs = ["lib"]
solc = "0.8.29"
optimizer = true
optimizer_runs = 200
ffi = true # To avoid setting --ffi in the test command
```

<Tip>
  See [CI/CD Integration](/credible/ci-cd-integration#foundry-profile-configuration) for complete Foundry profile configuration.
</Tip>

## Best Practices

1. **Start small** - Test with 10-20 blocks first to verify your setup
2. **Use paid RPC providers** - For block ranges over 1,000
3. **Check assertion failures** - Ensure `results.assertionFailures == 0`
4. **Run before deployment** - Validate behavior on real transactions
5. **Run manually** - Exclude from CI/CD to avoid long test runs
6. **Debug with single transactions** - Use `executeBacktestForTransaction` to investigate specific failures

**Learn More:**

* [Backtesting Reference](/credible/backtesting-reference) - Complete configuration details
* [Testing Assertions](/credible/testing-assertions) - Basic testing guide
* [Fuzz Testing](/credible/fuzz-testing) - Test with random inputs
* [Troubleshooting](/credible/troubleshooting) - Common errors and solutions
* [Balancer V2 Rate Manipulation Exploit](/assertions-book/previous-hacks/balancer-v2-stable-rate-exploit)