Richard Lin

I architect and deploy production AI systems

HackAI winner • AWS infrastructure specialist • ML pipeline designer

7 ML Experts
10 EC2 Instances
3 Security Groups

Frizzle's Rubric: Distributed AI Evaluation System

Click any component to explore technical details • Hover to see connections

Distributed AI System at Scale

Building Frizzle's Rubric required coordinating 7 specialized evaluation components across a secure, scalable infrastructure. The system needed to handle concurrent prompt evaluations while maintaining low latency and high availability.

ML Pipeline Design

Fine-tuned 5 DistilBERT models on custom datasets, achieving 94% accuracy on prompt clarity evaluation.

Infrastructure Security

Implemented VPC with bastion host architecture, enabling secure access to private subnet resources.

Async Orchestration

Designed timeout-resilient API coordination across microservices with graceful error handling.

Technical Problem Solving

SSH Agent Forwarding

Problem: Complex SSH access through bastion host to private instances

Solution: Configured ProxyJump with agent forwarding for seamless multi-hop SSH

SSH Configuration
Host orchestrator
    HostName 172.31.48.224
    ProxyJump frizzlesrubric-bastion
    ForwardAgent yes

Service Discovery

Problem: Coordinating API calls across 7 distributed services

Solution: Configuration-driven service discovery with hardcoded IPs for hackathon speed

Python Configuration
EXPERTS = {
    'clarity': '172.31.48.225:8003',
    'grammar': '172.31.48.226:8004',
    # ... 5 more experts
}

Cost Optimization

Problem: $3,905/month AWS costs unsustainable for demo

Solution: Analyzed cost breakdown and identified GPU instances as primary driver

  • GPU Instances: $3,033.60/month (77.7%)
  • Storage: $535.00/month (13.7%)
  • Networking: $222.20/month (5.7%)