Skip to main content

Redis Sidecar Documentation for ExFlow OnPrem Cache

Overview​

The Redis sidecar in ExFlow OnPrem Cache implements a distributed caching layer with intelligent fallback capabilities. It serves as a performance optimization and resilience mechanism for file operations, operating independently alongside the main application container.

Architecture Pattern​

Primary Functions​

1. File Metadata Caching​

Redis acts as a high-performance cache layer for file operations, storing frequently accessed metadata to reduce filesystem I/O.

What Gets Cached:

  • File Metadata: Size, content type, ETag, last modified dates
  • Container Information: Directory structure and file listings
  • Health Status: Connection diagnostics and performance metrics
  • Cache Keys Format: filerepo:{container}:{filename} (configurable prefix)

2. Intelligent Fallback System​

Your implementation includes sophisticated fallback logic:

Fallback Hierarchy:

  1. Primary: Redis distributed cache
  2. Secondary: In-memory cache (local process)
  3. Tertiary: Direct filesystem access (no caching)

Configuration​

Based on your appsettings.json, here are the key Redis sidecar settings:

Environment Variable Overrides:

  • REDIS_DISABLED=true - Forces in-memory cache
  • REDIS_CONNECTION=host:port - Override connection string
  • REDIS_CONNECT_TIMEOUT_MS=5000 - Adjust timeout
  • REDIS_RETRY=3 - Modify retry attempts

Performance Benefits​

Cache-Aside Pattern Implementation​

Your FileRepositoryService likely implements the cache-aside pattern:

  1. Read Operations:

    • Check Redis cache first
    • If miss, read from filesystem
    • Store result in Redis for future requests

  2. Write Operations:

    • Write to filesystem
    • Update/invalidate Redis cache
    • Ensure cache consistency

Expected Performance Improvements:​

  • Metadata Queries: 10-100x faster (Redis vs filesystem)
  • File Listings: Significant improvement for large directories
  • Reduced I/O: Fewer filesystem operations under load
  • Scalability: Multiple app instances share cache state

Operational Features​

Health Monitoring​

Your implementation includes comprehensive Redis health tracking:

Available Diagnostics:

  • /api/redis/status - Real-time Redis connection status
  • /api/health/ready - Overall system readiness including Redis
  • Connection failure/restoration timestamps
  • Error tracking and correlation IDs

Cache Invalidation Strategy​

Based on your file operations:

  • File Upload: Cache invalidated on successful write
  • File Delete: Cache entry removed immediately
  • Metadata Update: Cache refreshed with new values
  • TTL Expiration: Automatic cleanup of stale entries

Performance Issues:​

  • Monitor cache hit ratios via Redis logs
  • Check network latency between app and Redis
  • Verify Redis memory usage and eviction policies

Fallback Behavior:​

  • Application continues working with in-memory cache
  • Performance degrades but functionality maintained
  • Monitor logs for fallback activation

Security Considerations​

Network Security​

  • Redis typically runs on internal network only
  • No authentication required in sidecar pattern
  • Consider Redis AUTH for production environments

Data Sensitivity​

  • Only metadata cached (not file contents)
  • Cache keys don't contain sensitive information
  • Automatic expiration prevents stale data exposure

Monitoring & Observability​

Key Metrics to Track​

  • Redis connection uptime/downtime
  • Cache hit/miss ratios
  • Memory usage in Redis
  • Network latency to Redis
  • Fallback activation frequency

Log Analysis​

Your application provides rich logging for Redis operations:

  • Connection establishment/failures
  • Cache operations and performance
  • Fallback activation events
  • Health check results

Best Practices​

  1. Connection Management: Use connection pooling (handled by StackExchange.Redis)
  2. Error Handling: Implement circuit breaker patterns for Redis failures
  3. Monitoring: Set up alerts for Redis downtime and fallback activation
  4. Capacity Planning: Monitor Redis memory usage and plan for growth
  5. Backup Strategy: Redis persistence configured for cache durability

External Resources​

Here are some resources that explain the Redis sidecar concept:

This Redis sidecar implementation provides a robust, production-ready caching solution that enhances performance while maintaining system reliability through intelligent fallback mechanisms.