Agentic QA Engineer – Generative AI & Agentic Systems
- Location
- Dallas, Texas, United States
- Salary
- USD 50 / hour
- Company
- Akash pawar
Job description
• Role & Company / Project Overview Bulwark Management Consultants is hiring an Agentic QA Engineer focused on Generative AI, Agentic Systems, and Multi-Agent Testing for a long-term engagement. • Key Responsibilities & Daily Duties - Design and execute comprehensive QA strategies for Agentic AI and multi-agent systems. - Validate agent orchestration, tool calling, planner-executor loops, and inter-agent coordination. - Perform resilience, chaos, latency, load, and scalability testing across distributed AI systems. - Build evaluation frameworks for accuracy, safety, guardrails, and compliance. - Create reusable test artifacts, simulators, and synthetic datasets for evaluation. - Integrate automated tests into CI/CD pipelines and production observability/monitoring. • Required Qualifications, Skills & Experience - 7+ years of Software QA / Testing experience. - 2+ years working with AI/ML or LLM-based systems. - Hands-on experience testing Agentic / Multi-Agent architectures. - Strong Python programming skills. - Experience with LLM evaluation (exact/soft match, semantic similarity, guardrails, and prompt testing). - Distributed systems testing, latency profiling, and resiliency patterns. - Hands-on experience with LangChain, LangGraph, LlamaIndex, DSPy, OpenAI Agents, or similar frameworks. - Experience with CI/CD tools: GitHub Actions / Azure DevOps. - Observability and monitoring tools: OpenTelemetry, Prometheus/Grafana, Datadog, or similar. - Understanding of AI security, PII, compliance, and model safety. - Preferred skills: Multi-agent simulators, tooling latency emulation, MLOps, model versioning, evaluation pipelines, A/B experimentation for LLMs, AWS, serverless, containerization, cost/latency SLAs/SLOs for AI workloads, feature flags, and canary deployments. - Strong communication and leadership skills. - Eligibility: Valid US Driver's License required; open to USC, GC, GC EAD, H1B, and H4 EAD. • Working Hours, Shifts & Location Details - Location: Dallas, Texas, United States. - Work Mode: On-site 4–5 days per week. Candidates must be local to Dallas, TX and available for two rounds of face-to-face interviews. - Duration: Long Term. - Openings: 1. • Compensation & Benefits - Rate: $50 / Hour (USD). • Application Instructions - Send updated resume along with Full Name, Current Location, Visa Status, DL Status, Total Experience, Relevant Experience, Availability, and Expected Rate to the recruitment team.