AttestProtocol Documentation

Storage Architecture

AttestProtocol uses a hybrid storage architecture that combines on-chain immutability with off-chain scalability. This approach ensures cryptographic integrity while providing fast access and cost-effective operations.

Storage Components

On-Chain Storage

Immutable attestation recordsCore attestation data stored directly on blockchain networks for cryptographic integrity and decentralized verification.

Indexing Layer

Fast query optimizationHigh-performance indexing services that enable rapid attestation discovery and complex query operations.

IPFS Integration

Distributed content storageLarge attestation data and metadata stored on IPFS for decentralized, content-addressed retrieval.

Caching Systems

Performance accelerationMulti-tier caching for frequently accessed attestations and schema definitions.

Blockchain-Specific Storage

Stellar (Soroban) Storage

Stellar uses Soroban smart contracts for attestation storage:

// Soroban contract storage structure
#[contracttype]
pub struct Attestation {
    pub uid: String,
    pub schema_uid: String,
    pub attester: Address,
    pub subject: Address,
    pub data: String,
    pub timestamp: u64,
    pub expiration: u64,
    pub revoked: bool,
    pub reference: String,
}

Storage Characteristics:

Cost: ~0.00001 XLM per attestation
Finality: 3-5 seconds
Query Speed: Direct contract calls
Scalability: 1000+ TPS theoretical

Solana Storage

Solana uses Program Derived Addresses (PDAs) for attestation storage:

// Anchor account structure
#[account]
pub struct AttestationAccount {
    pub schema: Pubkey,
    pub attester: Pubkey,
    pub recipient: Pubkey,
    pub data: String,
    pub timestamp: i64,
    pub expiration_time: i64,
    pub revocable: bool,
    pub uid: String,
    pub revoked: bool,
}

Storage Characteristics:

Cost: ~0.002 SOL per attestation
Finality: 400ms average
Query Speed: RPC + PDA derivation
Scalability: 50,000+ TPS theoretical

Starknet Storage (Coming Soon)

Cairo-based storage with zero-knowledge capabilities:

// Cairo storage structure
struct Attestation {
    uid: felt252,
    schema_uid: felt252,
    attester: ContractAddress,
    subject: ContractAddress,
    data_hash: felt252,
    timestamp: u64,
    expiration: u64,
    revoked: bool,
}

Data Storage Patterns

Full On-Chain Storage

Store complete attestation data on-chain:

// Example: Basic identity verification
const attestation = await sdk.attest({
  schemaUID: 'identity-basic-v1',
  subject: userAddress,
  value: 'verified:true,level:basic,score:85,country:US,timestamp:1704067200',
  reference: 'kyc-session-123'
});

Pros:

Complete decentralization
Cryptographic integrity
No external dependencies

Cons:

Higher storage costs
Size limitations
Privacy concerns

Hash-Based Storage

Store data hashes on-chain with full data off-chain:

// Store only hash on-chain
const dataHash = sha256(JSON.stringify(userData));
const attestation = await sdk.attest({
  schemaUID: 'identity-private-v1',
  subject: userAddress,
  value: `dataHash:${dataHash},verified:true,timestamp:${Date.now()}`,
  reference: 'private-verification'
});

// Store full data on IPFS
const ipfsHash = await ipfs.add(JSON.stringify(userData));

Pros:

Lower on-chain costs
Enhanced privacy
Unlimited data size

Cons:

External dependencies
Data availability concerns
Additional complexity

Hybrid Storage

Combine public and private data:

// Public verification status + private details
const publicData = 'verified:true,level:enhanced,timestamp:1704067200';
const privateDataHash = sha256(JSON.stringify(sensitiveData));

const attestation = await sdk.attest({
  schemaUID: 'kyc-hybrid-v1',
  subject: userAddress,
  value: `${publicData},privateHash:${privateDataHash}`,
  reference: 'hybrid-verification'
});

Indexing and Querying

Horizon Indexer (Stellar)

AttestProtocol’s custom indexer for Stellar attestations:

// Query attestations by subject
const attestations = await horizonClient.getAttestations({
  subject: userAddress,
  schema: 'kyc-basic-v1',
  status: 'active'
});

// Query by authority
const authorityAttestations = await horizonClient.getAttestations({
  attester: authorityAddress,
  limit: 100,
  order: 'desc'
});

Features:

Real-time indexing
Complex query support
Aggregation capabilities
Historical data access

Solana Indexing

Using Solana’s account scanning and filtering:

// Get all attestations for a subject
const attestations = await connection.getProgramAccounts(PROGRAM_ID, {
  filters: [
    {
      memcmp: {
        offset: 8 + 32, // Skip discriminator and schema
        bytes: userAddress.toBase58()
      }
    }
  ]
});

Optimization Strategies:

PDA-based organization
Account scanning with filters
Custom RPC endpoints
Geyser plugin integration

Custom Query APIs

High-level query interfaces:

// Complex attestation queries
const results = await queryAPI.search({
  subjects: [userAddress1, userAddress2],
  schemas: ['kyc-basic-v1', 'kyc-enhanced-v1'],
  authorities: [trustedAuthority],
  dateRange: {
    from: new Date('2024-01-01'),
    to: new Date('2024-12-31')
  },
  status: 'active',
  orderBy: 'timestamp',
  order: 'desc',
  limit: 50
});

Performance Optimization

Caching Strategies

Multi-tier caching for optimal performance:

class AttestationCache {
  private l1Cache = new Map(); // In-memory
  private l2Cache = redis; // Redis cluster
  private l3Cache = database; // Persistent storage

  async getAttestation(key: string) {
    // L1: Memory cache (fastest)
    if (this.l1Cache.has(key)) {
      return this.l1Cache.get(key);
    }

    // L2: Redis cache (fast)
    const l2Result = await this.l2Cache.get(key);
    if (l2Result) {
      this.l1Cache.set(key, l2Result);
      return l2Result;
    }

    // L3: Database/blockchain (slowest)
    const l3Result = await this.fetchFromBlockchain(key);
    if (l3Result) {
      await this.l2Cache.setex(key, 300, l3Result); // 5 min TTL
      this.l1Cache.set(key, l3Result);
    }

    return l3Result;
  }
}

Batch Operations

Optimize for high-throughput scenarios:

// Batch attestation creation
const batchResults = await sdk.batchAttest([
  {
    schemaUID: 'user-verification-v1',
    subject: user1Address,
    value: 'verified:true,score:90',
    reference: 'batch-001'
  },
  {
    schemaUID: 'user-verification-v1',
    subject: user2Address,
    value: 'verified:true,score:85',
    reference: 'batch-002'
  }
  // ... more attestations
]);

// Batch retrieval
const attestations = await sdk.batchFetch([
  { schemaUID: 'kyc-v1', subject: user1Address },
  { schemaUID: 'kyc-v1', subject: user2Address },
  { schemaUID: 'reputation-v1', subject: user1Address }
]);

Connection Pooling

Optimize blockchain connections:

class ConnectionPool {
  private stellar: StellarSDK.Server[] = [];
  private solana: Connection[] = [];
  private currentStellar = 0;
  private currentSolana = 0;

  constructor(config: PoolConfig) {
    // Initialize connection pools
    for (let i = 0; i < config.stellar.poolSize; i++) {
      this.stellar.push(new StellarSDK.Server(config.stellar.url));
    }
    
    for (let i = 0; i < config.solana.poolSize; i++) {
      this.solana.push(new Connection(config.solana.url));
    }
  }

  getStellarConnection(): StellarSDK.Server {
    const connection = this.stellar[this.currentStellar];
    this.currentStellar = (this.currentStellar + 1) % this.stellar.length;
    return connection;
  }
}

Data Lifecycle Management

Archival Strategies

Manage long-term data storage:

interface ArchivalPolicy {
  activeRetention: number; // Days to keep in active storage
  archiveRetention: number; // Days to keep in archive
  compressionEnabled: boolean;
  redundancyLevel: number;
}

class DataLifecycleManager {
  async archiveOldAttestations(cutoffDate: Date) {
    const oldAttestations = await this.queryOldAttestations(cutoffDate);
    
    for (const attestation of oldAttestations) {
      // Compress and move to archive storage
      const compressed = await this.compressAttestation(attestation);
      await this.moveToArchive(compressed);
      
      // Update index to point to archive location
      await this.updateIndex(attestation.uid, 'archived');
    }
  }

  async retrieveArchivedAttestation(uid: string) {
    const archiveLocation = await this.getArchiveLocation(uid);
    const compressed = await this.retrieveFromArchive(archiveLocation);
    return await this.decompressAttestation(compressed);
  }
}

Pruning and Cleanup

Remove expired or revoked attestations:

class StorageManager {
  async cleanupExpiredAttestations() {
    const now = Date.now() / 1000;
    
    // Find expired attestations
    const expired = await this.queryExpired(now);
    
    for (const attestation of expired) {
      // Mark as expired in index
      await this.updateStatus(attestation.uid, 'expired');
      
      // Optionally remove from active storage
      if (this.config.removeExpired) {
        await this.removeFromStorage(attestation.uid);
      }
    }
  }

  async cleanupRevokedAttestations(olderThan: Date) {
    const revoked = await this.queryRevoked(olderThan);
    
    for (const attestation of revoked) {
      await this.archiveAttestation(attestation);
      await this.removeFromActiveStorage(attestation.uid);
    }
  }
}

Privacy and Encryption

Field-Level Encryption

Encrypt sensitive fields before storage:

class EncryptedAttestationStorage {
  private encryptionKey: CryptoKey;

  async storeAttestation(data: AttestationData) {
    // Encrypt sensitive fields
    const encryptedData = {
      ...data,
      personalInfo: await this.encrypt(data.personalInfo),
      financialData: await this.encrypt(data.financialData),
      // Keep non-sensitive fields unencrypted for queries
      verified: data.verified,
      timestamp: data.timestamp
    };

    return await this.sdk.attest({
      schemaUID: 'encrypted-kyc-v1',
      subject: data.subject,
      value: JSON.stringify(encryptedData),
      reference: data.reference
    });
  }

  async retrieveAttestation(uid: string) {
    const attestation = await this.sdk.fetchAttestation(uid);
    
    if (attestation.data) {
      const data = JSON.parse(attestation.data.value);
      
      // Decrypt sensitive fields
      return {
        ...data,
        personalInfo: await this.decrypt(data.personalInfo),
        financialData: await this.decrypt(data.financialData)
      };
    }
  }
}

Zero-Knowledge Storage

Store proofs without revealing data:

class ZKAttestationStorage {
  async createZKAttestation(claims: any, proof: ZKProof) {
    // Store only the proof and public outputs
    const attestation = await this.sdk.attest({
      schemaUID: 'zk-proof-v1',
      subject: claims.subject,
      value: `proofHash:${proof.hash},publicOutputs:${proof.publicOutputs},verified:true`,
      reference: proof.sessionId
    });

    // Store full proof off-chain
    await this.storeProofOffChain(proof);
    
    return attestation;
  }

  async verifyZKAttestation(uid: string) {
    const attestation = await this.sdk.fetchAttestation(uid);
    const proof = await this.retrieveProofOffChain(attestation.data.reference);
    
    return await this.verifyProof(proof);
  }
}

Cross-Chain Storage

Multi-Chain Synchronization

Keep attestations synchronized across networks:

class CrossChainStorage {
  private chains = ['stellar', 'solana', 'starknet'];

  async createCrossChainAttestation(data: AttestationData) {
    const results = await Promise.allSettled(
      this.chains.map(async (chain) => {
        const sdk = await this.getSDK(chain);
        return await sdk.attest(data);
      })
    );

    // Handle partial failures
    const successful = results
      .filter(r => r.status === 'fulfilled')
      .map(r => (r as PromiseFulfilledResult<any>).value);

    if (successful.length === 0) {
      throw new Error('Failed to create attestation on any chain');
    }

    return {
      successful,
      failed: results.filter(r => r.status === 'rejected').length
    };
  }

  async syncAttestations() {
    for (const sourceChain of this.chains) {
      const sourceAttestations = await this.getUnsynced(sourceChain);
      
      for (const attestation of sourceAttestations) {
        await this.replicateToOtherChains(attestation, sourceChain);
      }
    }
  }
}

Bridge Attestations

Create attestations that reference cross-chain data:

// Bridge attestation linking Stellar and Solana data
const bridgeAttestation = await stellarSDK.attest({
  schemaUID: 'cross-chain-bridge-v1',
  subject: userAddress,
  value: `sourceChain:solana,sourceUID:${solanaAttestationUID},verified:true,timestamp:${Date.now()}`,
  reference: `bridge-${sourceUID}`
});

Backup and Recovery

Distributed Backups

Implement robust backup strategies:

class BackupManager {
  private backupTargets = [
    { type: 'ipfs', endpoint: 'https://ipfs.io' },
    { type: 'arweave', endpoint: 'https://arweave.net' },
    { type: 's3', endpoint: 'https://s3.amazonaws.com' }
  ];

  async backupAttestation(attestation: Attestation) {
    const backupData = {
      attestation,
      metadata: {
        backupDate: new Date(),
        version: '1.0',
        checksum: this.calculateChecksum(attestation)
      }
    };

    const results = await Promise.allSettled(
      this.backupTargets.map(target => 
        this.storeToTarget(target, backupData)
      )
    );

    return {
      success: results.filter(r => r.status === 'fulfilled').length,
      total: results.length,
      locations: results
        .filter(r => r.status === 'fulfilled')
        .map(r => (r as any).value.location)
    };
  }

  async recoverAttestation(uid: string) {
    for (const target of this.backupTargets) {
      try {
        const backup = await this.retrieveFromTarget(target, uid);
        
        // Verify integrity
        if (this.verifyChecksum(backup)) {
          return backup.attestation;
        }
      } catch (error) {
        console.warn(`Recovery failed from ${target.type}:`, error);
      }
    }

    throw new Error(`Unable to recover attestation ${uid} from any backup`);
  }
}

Disaster Recovery

Plan for catastrophic failures:

class DisasterRecoveryManager {
  async createRecoverySnapshot() {
    const snapshot = {
      timestamp: Date.now(),
      schemas: await this.exportAllSchemas(),
      authorities: await this.exportAllAuthorities(),
      attestations: await this.exportActiveAttestations(),
      indices: await this.exportIndexes()
    };

    // Store snapshot across multiple locations
    await Promise.all([
      this.storeToIPFS(snapshot),
      this.storeToArweave(snapshot),
      this.storeToS3(snapshot)
    ]);

    return snapshot;
  }

  async restoreFromSnapshot(snapshotHash: string) {
    const snapshot = await this.retrieveSnapshot(snapshotHash);
    
    // Restore in order: schemas, authorities, attestations
    await this.restoreSchemas(snapshot.schemas);
    await this.restoreAuthorities(snapshot.authorities);
    await this.restoreAttestations(snapshot.attestations);
    await this.rebuildIndices(snapshot.indices);
  }
}

Monitoring and Analytics

Storage Metrics

Track storage health and performance:

interface StorageMetrics {
  totalAttestations: number;
  storageUsed: number; // bytes
  averageSize: number; // bytes per attestation
  writeLatency: number; // ms
  readLatency: number; // ms
  cacheHitRate: number; // percentage
  errorRate: number; // percentage
  chainDistribution: Record<string, number>;
}

class StorageMonitor {
  async collectMetrics(): Promise<StorageMetrics> {
    return {
      totalAttestations: await this.countAttestations(),
      storageUsed: await this.calculateStorageUsed(),
      averageSize: await this.calculateAverageSize(),
      writeLatency: await this.measureWriteLatency(),
      readLatency: await this.measureReadLatency(),
      cacheHitRate: await this.calculateCacheHitRate(),
      errorRate: await this.calculateErrorRate(),
      chainDistribution: await this.getChainDistribution()
    };
  }

  async alertOnThresholds(metrics: StorageMetrics) {
    if (metrics.writeLatency > 5000) { // 5 seconds
      await this.sendAlert('High write latency detected');
    }

    if (metrics.errorRate > 0.05) { // 5%
      await this.sendAlert('High error rate detected');
    }

    if (metrics.cacheHitRate < 0.8) { // 80%
      await this.sendAlert('Low cache hit rate');
    }
  }
}

Best Practices

Storage Design

Choose appropriate storage patterns based on data sensitivity
Implement proper data lifecycle management
Use caching strategically to improve performance
Plan for cross-chain synchronization early

Performance

Batch operations when possible
Implement multi-tier caching
Use connection pooling for blockchain clients
Monitor and optimize query patterns

Security

Encrypt sensitive data before storage
Implement proper access controls
Use checksums for data integrity
Plan for disaster recovery scenarios

Scalability

Design for horizontal scaling
Implement data partitioning strategies
Use appropriate indexing for query patterns
Plan archive and cleanup procedures

Next Steps

SDK Storage Methods

Explore storage-related SDK methods and configuration

Indexing Guide

Learn to build custom indexing solutions

Performance Optimization

Optimize storage for production workloads

Backup Strategies

Implement robust backup and recovery systems

Introduction

Getting Started

Core Concepts

Stellar Architecture

Workflow Guides

​Storage Architecture

​Storage Components

On-Chain Storage

Indexing Layer

IPFS Integration

Caching Systems

​Blockchain-Specific Storage

​Stellar (Soroban) Storage

​Solana Storage

​Starknet Storage (Coming Soon)

​Data Storage Patterns

​Full On-Chain Storage

​Hash-Based Storage

​Hybrid Storage

​Indexing and Querying

​Horizon Indexer (Stellar)

​Solana Indexing

​Custom Query APIs

​Performance Optimization

​Caching Strategies

​Batch Operations

​Connection Pooling

​Data Lifecycle Management

​Archival Strategies

​Pruning and Cleanup

​Privacy and Encryption

​Field-Level Encryption

​Zero-Knowledge Storage

​Cross-Chain Storage

​Multi-Chain Synchronization

​Bridge Attestations

​Backup and Recovery

​Distributed Backups

​Disaster Recovery

​Monitoring and Analytics

​Storage Metrics

​Best Practices

​Next Steps

SDK Storage Methods

Indexing Guide

Performance Optimization

Backup Strategies

Storage Architecture

Storage Components

Blockchain-Specific Storage

Stellar (Soroban) Storage

Solana Storage

Starknet Storage (Coming Soon)

Data Storage Patterns

Full On-Chain Storage

Hash-Based Storage

Hybrid Storage

Indexing and Querying

Horizon Indexer (Stellar)

Solana Indexing

Custom Query APIs

Performance Optimization

Caching Strategies

Batch Operations

Connection Pooling

Data Lifecycle Management

Archival Strategies

Pruning and Cleanup

Privacy and Encryption

Field-Level Encryption

Zero-Knowledge Storage

Cross-Chain Storage

Multi-Chain Synchronization

Bridge Attestations

Backup and Recovery

Distributed Backups

Disaster Recovery

Monitoring and Analytics

Storage Metrics

Best Practices

Next Steps