
Abdelrahman Youssef. Applied AI Engineer. I evaluate frontier models and build agentic systems. Currently contracting at Outlier and Alignerr. Previously led the engineering team behind a production platform on AWS serving 12,000 users.
Projects
An agent harness that runs AI coding workers against a task backlog. Each iteration spawns a worker in a fresh git worktree, verifies the output, and merges back. State is persisted before every action — a crash at any point resumes cleanly. Roadblocks escalate via Discord.
Three-route agent orchestrated via a LangGraph state machine with dynamic re-ranking: FAISS vector store (100K+ parent-child chunks, BGE-m3 embeddings, 13 business datasets), Python data analysis with self-correcting code generation (3-retry LLM critique loop), and Tavily web search fallback. Llama 3.2 Vision for multi-modal document understanding; built-in hallucination detection, document grading, and answer validation gates. Evaluated retrieval quality via RAGAS (faithfulness, answer relevancy, context precision, context recall). Containerized with Docker + FastAPI, deployed on AWS EC2 with automated health checks via GitHub Actions CI/CD. Built for consulting-firm knowledge-assistant use case; production-graded: containerized, CI/CD-deployed, instrumented with health checks.
Automated benchmarking framework evaluating 10+ LLMs (GPT-4o, Claude 3.5, Gemini Ultra, Llama 3 family, Mistral variants) against Stockfish 16 across 25 tactical chess metrics: mate-in-1 accuracy, forcing moves, blunder rates, style consistency, stagnation detection, tactic prioritization. Dual-signal evaluation — outcome (win-rate) + reasoning quality (move quality vs. engine) — enabling cross-model-family capability gap analysis. Chess as reasoning proxy: deterministic ground truth (Stockfish), quantifiable sub-goals (mate-in-1, forcing lines), measurable style — separates outcome quality from reasoning quality, a distinction most benchmarks conflate. Open-sourced with a reproducible analysis notebook; demonstrates evaluation infrastructure design without relying on existing benchmarks.
1,000 simulated cars trained in parallel using a feed-forward neural network (tanh neurons, implemented from first principles in JavaScript for browser-side real-time simulation) and a genetic algorithm with mutation-based evolution. Collision detection via segment-intersection math; emergent lane-following and collision-avoidance behaviors discovered without any hard-coded rules. Demonstrates core DL fluency (forward pass, backpropagation, gradient descent, mutation-selection loops) independent of framework convenience — the same primitives underpinning production PyTorch/TF workflows. Live browser demo.
Skills
Eval & RAG
- RLHF rubric design
- LangGraph
- LangChain
- FAISS (BGE-m3)
- RAGAS
- Cohen's κ calibration
ML & Stats
- PyTorch
- HuggingFace
- XGBoost
- scikit-learn
- SHAP
- statsmodels (panel, clustered SE, DiD)
- Causal inference
Production
- Docker
- GitHub Actions
- AWS (EC2/S3/ECR)
- Nginx
- Linux/RHEL
Languages
- Python (advanced)
- TypeScript
- React
- Node.js
- SQL
Databases
- PostgreSQL
- MongoDB
- FAISS
- SQLite
Timeline
AI Evaluation Engineer — ML & Multimodal (Independent Contractor)
Alignerr
Authored ML research tasks with deterministic graders whose scores became RL training signals; designed problems that stumped frontier models across prediction and policy-training paradigms. Expanded into multimodal evaluation (image classification, audio transcription quality, multimodal agent routing); developed failure-mode detection across modality boundaries.
Jan 2025 – PresentAI Evaluation Engineer — Coding Models (Independent Contractor)
Outlier
5,000+ RLHF evaluations across 100+ frontier models (OpenAI, Meta, Microsoft, Nvidia) at 93% acceptance (96% with retries) on coding, math, agentic tool-use, reasoning, image, and audio workloads. Designed multi-axis rubrics (correctness, reasoning coherence, instruction-following) consumed directly by downstream fine-tuning pipelines; raised inter-annotator agreement (Cohen's κ) from 0.65 to 0.85. Judged model function-calling end-to-end — missing/malformed API calls, wrong outputs, tool-factuality errors, hallucinated tool results.
Feb 2024 – PresentSoftware Engineer (Production Platform)
University of Rochester, Information Technology
Built and owned a MERN platform on AWS behind Nginx, architected for the entire student body (~12,000 students) — full-stack from database schema to React UI. Internal progression: Help Desk Assistant → Software Engineer. Slashed incident MTTR from 30–40 minutes to <5 minutes via version-pinning revert (Docker image + git SHA tagging → ALB target group swap). Cut page load from 3–8s to <300ms (p95) with server-side pagination, lazy loading, and in-memory caching. Integrated Shibboleth SSO campus-wide (login tickets down 80%+) and reduced AWS spend ~25% via EC2/S3 rightsizing. Hired and mentored 5 of 6 engineers; introduced Jest testing and formalized on-call rotation.
Aug 2019 – Dec 2023M.S. in AI for Business (50% merit scholarship)
Simon Business School, University of Rochester
Graduate coursework: Causal Inference, Experimental Design, Big Data Methods, ML for Business. Shipped ML projects including HELOC credit-risk 6-model pipeline (Streamlit + SHAP), CNN on CelebA (40K+ images, 116MB checkpoint), from-scratch Transformer NMT (489MB checkpoint), VAE + smile-vector, 3-tier SQL warehouse + Tableau. Graduating May 2026.
May 2026B.S. in Computer Science, Business Minor
University of Rochester
Coursework: Design & Analysis of Efficient Algorithms, Programming Language Design & Implementation, AI, Web Programming, Data Structures & Algorithms, Mobile App Development, Database Systems, Computer Organization. Workshop Leader and Teaching Assistant for CS and Business-minor courses.
Dec 2024Certifications
Anthropic
- Building with the Claude API
- AI Fluency: Framework & Foundations
- Claude 101
- Claude Code 101
- Claude Platform 101
- Foundations of Data Science
- The Power of Statistics
- Translating Data Into Insights