What this objective covers · 7% of the exam
Determine approaches to evaluate FM performance (for example, human-in-the-loop evaluation, benchmark datasets, Amazon Bedrock Model Evaluation).; Identify relevant metrics to assess FM performance (for example, Recall-Oriented Understudy for Gisting Evaluation [ROUGE], Bilingual Evaluation Understudy [BLEU], BERTScore, LLM-as-a-judge).; Determine whether an FM effectively meets business objectives (for example, productivity, user engagement, task engineering).; Identify approaches to evaluate the performance of applications built with FM (for example, RAG, agents, workflows).; Identify business objective alignment metrics for AI applications (for example, task completion rate, user satisfaction, cost per interaction).
- Determine approaches to evaluate FM performance (for example, human-in-the-loop evaluation, benchmark datasets, Amazon Bedrock Model Evaluation).
- Identify relevant metrics to assess FM performance (for example, Recall-Oriented Understudy for Gisting Evaluation [ROUGE], Bilingual Evaluation Understudy [BLEU], BERTScore, LLM-as-a-judge).
- Determine whether an FM effectively meets business objectives (for example, productivity, user engagement, task engineering).
- Identify approaches to evaluate the performance of applications built with FM (for example, RAG, agents, workflows).
- Identify business objective alignment metrics for AI applications (for example, task completion rate, user satisfaction, cost per interaction).