01
Evaluation dataset
Make this concrete enough that another engineer can understand what you built and why.
CAREER PROJECT · Advanced · 2–3 weekends
RAG Evaluation Workbench
Build a retrieval-augmented generation app that measures quality instead of stopping at a chat demo.
LLM integrationRetrievalEvaluationPrompt designData pipelinesObservability
DEFINITION OF DONE
Leave artifacts a reviewer can inspect.
02
Retrieval pipeline
Make this concrete enough that another engineer can understand what you built and why.
03
Quality dashboard
Make this concrete enough that another engineer can understand what you built and why.
04
Failure analysis
Make this concrete enough that another engineer can understand what you built and why.
05
Cost/latency notes
Make this concrete enough that another engineer can understand what you built and why.
RESUME / PORTFOLIO FRAMING
Describe what you actually did.
Implemented a retrieval-augmented generation pipeline with repeatable evaluation, failure analysis and latency/cost instrumentation.
Use this as a starting structure, then replace it with your real implementation details, metrics and constraints. Do not claim outcomes you did not measure.
STRETCH GOALS
Go deeper if the core is already solid.
Reranking
Add only if it strengthens the engineering story instead of delaying a finished, deployable core.
Hybrid search
Add only if it strengthens the engineering story instead of delaying a finished, deployable core.
Tracing
Add only if it strengthens the engineering story instead of delaying a finished, deployable core.
A/B prompt evaluation
Add only if it strengthens the engineering story instead of delaying a finished, deployable core.