Skip to main content

System Architecture

This guide provides a comprehensive overview of Agenta's system architecture, explaining each component's role, responsibilities, and how they work together to deliver AI application evaluation and management capabilities.

System Overview​

Agenta follows a modern microservices architecture with clear separation of concerns. The system is designed for scalability, maintainability, and ease of deployment using Docker containers.

                                    ┌─────────────────────────────────────┐
                                    │              Users                  │
                                    │    (Developers, AI Engineers)       │
                                    └─────────────────┬───────────────────┘
                                                      │
                                    ┌─────────────────▼───────────────────┐
                                    │        Load Balancer/Proxy          │
                                    │         (Traefik/Nginx)             │
                                    │      Handles SSL, Routing           │
                                    └─────────────┬───────────────────────┘
                                                  │
                    ┌─────────────────────────────┼─────────────────────────────┐
                    │                             │                             │
                    ▼                             ▼                             ▼
          ┌─────────────────┐         ┌─────────────────┐         ┌─────────────────┐
          │   Frontend      │         │    Backend      │         │   AI Services   │
          │   (Web UI)      │◄────────►    (API)        │◄────────►  (Completion)   │
          │                 │         │                 │         │  (Chat)         │
          │ • React App     │         │ • FastAPI       │         │ • LLM Providers │
          │ • User Interface│         │ • Business Logic│         │ • Model Serving │
          │ • Playground    │         │                 │         │                 │
          └─────────────────┘         └─────────┬───────┘         └─────────────────┘
                    │                           │
                    │                           ▼
                    │                 ┌─────────────────┐
                    │                 │     Worker      │
                    │                 │  (Background)   │
                    │                 │                 │
                    │                 │ • TaskIQ Tasks  │
                    │                 │ • Async Jobs    │
                    │                 │ • Evaluations   │
                    │                 └─────────┬───────┘
                    │                           │
                    ▼                           ▼
          ┌───────────────────────────────────────────────────────────────────┐
          │                       Infrastructure Layer                        │
          │                                                                   │
          │ ┌───────────────────┐ ┌───────────────────┐ ┌───────────────────┐ │
          │ │   PostgreSQL      │ │      Redis        │ │   SuperTokens     │ │
          │ │                   │ │                   │ │                   │ │
          │ │• Core DB          │ │• Task Queue       │ │• Auth             │ │
          │ │• Tracing DB       │ │  (TaskIQ)         │ │                   │ │
          │ │                   │ │• Caching          │ │                   │ │
          │ │                   │ │• Sessions         │ │                   │ │
          │ └───────────────────┘ └───────────────────┘ └───────────────────┘ │
          └───────────────────────────────────────────────────────────────────┘

Frontend Components​

Web UI (NextJS Application)​

  • Technology: React, TypeScript, Next.js
  • Port: 3000 (internal)
  • Purpose: Primary user interface for Agenta platform

Key Responsibilities:

  • User Interface: Provides intuitive web interface for application management
  • Playground: Interactive environment for testing and evaluating LLM applications
  • Evaluation Dashboard: Visualizations and metrics for application performance
  • Application Management: Create, configure, and deploy AI applications
  • User Authentication: Login, registration, and session management

Backend Components​

API Service (FastAPI)​

  • Technology: Python, FastAPI, SQLAlchemy
  • Port: 8000 (internal)
  • Purpose: Core business logic and API endpoints

Key Responsibilities:

  • REST API: Provides RESTful endpoints for frontend and external integrations
  • Business Logic: Implements core platform functionality
  • Data Management: Handles CRUD operations for applications, evaluations, experiments, etc
  • Authentication: Integrates with SuperTokens for user authentication
  • Application Orchestration: Manages application lifecycle and deployment
  • Evaluation Management: Coordinates evaluation runs and result collection

Worker Service (TaskIQ)​

  • Technology: Python, TaskIQ (task queue library), Redis
  • Purpose: Background task processing and async operations

Key Responsibilities:

  • Evaluation Execution: Runs AI model evaluations asynchronously
  • Data Processing: Batch processing of evaluation results
  • Scheduled Tasks: Periodic maintenance and cleanup operations
  • Resource Management: Manages compute resources for evaluations
  • Job Monitoring: Tracks task progress and handles failures

TaskIQ Integration:

  • Broker: Uses Redis Streams via RedisStreamBroker for task distribution
  • Task Registration: Evaluation tasks registered with the broker at startup
  • Execution: Workers consume tasks from Redis and execute them asynchronously

AI Services​

Key Responsibilities:

  • LLM Integration: Connects to various LLM providers (OpenAI, Anthropic, etc.)
  • Prompt Processing: Handles prompt templates and variable substitution
  • Response Generation: Manages LLM API calls and response handling
  • Provider Abstraction: Unified interface across different LLM providers
  • Error Handling: Robust error handling for LLM API failures

Completion Service​

  • Technology: Python, FastAPI
  • Port: 80 (internal)
  • Purpose: LLM completion and text generation

Chat Service​

  • Technology: Python, FastAPI
  • Port: 80 (internal)
  • Purpose: Conversational AI and chat-based interactions

Infrastructure Services​

PostgreSQL (Database)​

  • Technology: PostgreSQL 16
  • Port: 5432
  • Purpose: Primary data storage

Databases:

  • Core Database: Application data, Datasets, Evaluations, Users & Profiles, etc.
  • Tracing Database: Execution traces and performance metrics
  • SuperTokens Database: Authentication and user management data

Redis (Task Queue, Caching & Sessions)​

  • Technology: Redis
  • Port: 6379
  • Purpose: Task queue, caching, pub/sub, streams

Use Cases:

  • Task Queue: TaskIQ broker for background job distribution and processing
  • Application Caching: Frequently accessed data
  • Session Storage: User sessions and temporary data
  • Task Results: TaskIQ task results and status
  • Real-time Data: Live updates and notifications
  • Rate Limiting: API rate limit counters

SuperTokens (Authentication)​

  • Technology: SuperTokens
  • Port: 3567
  • Purpose: Authentication and user management

Features:

  • User Authentication: Login/logout, password management
  • Session Management: Secure session handling with JWT
  • OAuth Integration: Google, and GitHub
  • User Management: User registration, profile management

Service Dependencies​

Frontend Dependencies​

Web UI depends on:
├── API Service (primary backend)
├── AI Services (direct calls for playground)
└── Authentication (SuperTokens via API)

Backend Dependencies​

API Service depends on:
├── PostgreSQL (data persistence)
├── Redis (task queue, caching, sessions)
├── SuperTokens (authentication)
└── Worker (async task execution)

Worker Dependencies​

Worker depends on:
├── Redis (task queue via TaskIQ)
├── PostgreSQL (data access)
└── AI Services (evaluation execution)