Technical & Professional Requirements:
Automation Stack: High proficiency in Python (for AI testing) and framework automation (PyTest, Selenium, or Robot Framework).
Cloud Infrastructure: Strong hands-on experience with Azure or AWS, specifically regarding networking, scaling, and serverless reliability.
AI/ML Understanding: Understanding of Prompt Engineering and how to evaluate AI model outputs (RAG evaluation, ROUGE/BLEU scores, or custom LLM-benchmarks).
Monitoring Tools: Experience with Grafana, Prometheus, or native cloud monitoring tools to build real-time reliability dashboards.
FinOps Awareness: Ability to identify "expensive" failing tests or inefficient cloud resource usage during the testing phase.
Recommended Skillset & Tools:
Languages: Python (Mandatory), Bash scripting.
Tools: GitHub Actions (CI/CD), Terraform (reading/validating), K6 or JMeter (Performance).
AI Frameworks: DeepEval, Ragas, or LangSmith (for automated AI evaluation).