Agentic Engineer

Agentic Enginner

Dubai

Production agentic AI systems, LLM orchestration, MCP/A2A tool connectivity, RAG, evaluation, observability, safety and cost optimisation

• Design, build and operate production-grade agentic AI systems that can reason, use tools, maintain state, retrieve knowledge, execute multi-step workflows and escalate safely to humans when required.
• Select and integrate appropriate LLMs and agent frameworks across Azure OpenAI, Anthropic, open models and provider-native agent SDKs, balancing latency, cost, quality, data residency and compliance needs.
• Engineer reliable tool-use patterns including function calling, structured outputs, MCP servers, API wrappers, permissions, retries, timeouts, sandboxing and audit trails.
• Implement retrieval, memory and context-engineering patterns including RAG pipelines, hybrid search, re-ranking, short-term and long-term memory, summarisation and context budgeting.
• Own agent evaluation, safety and observability: automated evals, golden datasets, red-team testing, prompt-injection defences, PII controls, traceability, dashboards and production feedback loops.
• Optimise performance and commercial viability through token budgeting, prompt caching, model routing, batching, cost monitoring and continuous improvement of agent success rates.

Agent Architecture & Design

• Define agent workflows: What are the agent's goals? What tools does it need (APIs, databases, search)? What's the decision loop?
• Choose orchestration framework: LangChain, LlamaIndex, Azure Semantic Kernel, or custom agent loop (depends on complexity)
• Design tool interfaces: What functions can agents call? What parameters? What's the expected response format? (Function calling / structured output)
• Implement error handling: What if a tool fails? Can the agent recover? When should it escalate to human?
• Design multi-turn conversations: Context window management (summarize old turns, drop less relevant context); prevent infinite loops

LLM Selection & Prompting

• Evaluate models for use case: Latency requirements (Claude 3.5 Sonnet is fast for reasoning; GPT-4o for multimodal), cost per 1M tokens, context window (4K vs. 200K)
• Write system prompts: Clear role definition, task boundaries, safety guidelines (e.g., "Never execute user code directly"; "Always verify customer identity")
• Implement few-shot prompting: Provide examples of desired behavior; show edge case handling
• Implement chain-of-thought: Ask agents to reason step-by-step before answering; improves accuracy on complex tasks
• Test prompt robustness: Does it handle jailbreak attempts? Adversarial inputs? Different languages (Arabic for UAE)?

Tool Integration & Function Calling

• Define tool schemas: Functions agents can call (search database, fetch customer data, send email, make payment)
• Implement tool wrappers: Validate inputs (prevent SQL injection, large data fetches), execute safely in sandboxed environment, return structured responses
• Implement guardrails: Rate limit tool calls (prevent denial of service), check permissions (agent can't access customer data it doesn't own), audit logging
• Handle tool failures: Retry logic, fallback tools, error messages that help agent understand what went wrong
• Optimize tool calls: Batch calls if possible (fewer API round-trips), cache results (same question asked multiple times)

Prompt Caching & Cost Optimization

• Implement prompt caching (Azure OpenAI, Claude Prompt Caching): Reuse long context (documents, code) without re-processing; save :90% of tokens for cached portions
• Batch requests: If processing multiple documents/queries, batch them into single API call (if semantics allow)
• Use cheaper models for certain tasks: GPT-4o Turbo for complex reasoning, GPT-4o Mini for simple classification
• Implement token counting: Estimate costs before running agents; flag agents that are exceeding budget
• Monitor API costs: Track spend by agent, by model, by use case; optimize high-cost agents
• Safety & Guardrails
• Implement input validation: Filter adversarial prompts, prompt injection attempts
• Implement output filtering: Ensure agent doesn't leak PII, corporate secrets, or biased content
• Handle refusals: Agent refuses to perform unsafe action; provide clear explanation to user ("I can't modify payment history without approval")
• Audit logging: Log all agent interactions (user query, agent reasoning, tools called, final response) for compliance/debugging
• Implement human-in-the-loop: For sensitive actions (deleting data, transferring funds), require human approval before execution

Performance & Observability

• Monitor agent latency: Time from user query to response; breakdown by LLM call, tool execution, parsing
• Track accuracy metrics: If possible, grade agent outputs (correct answer? safe? complete?); compare model versions
• Implement observability: Log structured data (timestamps, tokens used, tool calls, errors) to Azure Application Insights or ELK
• Create dashboards: Agent success rate, error rate, average latency, cost per query
• Debug failures: Detailed logs of reasoning steps, function inputs/outputs; be able to replay conversations for investigation

Collaboration & Testing

• Work with Backend engineers: Define API contracts for agent function calling; ensure APIs return agent-friendly responses
• Work with QA Guardian: Design agent test scenarios (happy path, edge cases, adversarial inputs); create synthetic datasets
• Work with Data Factory Engineer: Request features for agents (embeddings for semantic search, historical data for context)
• Conduct code reviews: Review prompts, tool definitions, safety guardrails for risks
• Document agent behavior: What can the agent do? What are its limitations? When does it escalate to human?
• Production deployment: Package and deploy agents as reliable services using CI/CD, environment separation, secrets management, feature flags, canary releases, rollback plans and provider failover where needed. Build and maintain automated evaluation suites covering task completion, factuality, tool accuracy, safety, latency, cost, regression detection and human review workflows.

Person:

• 4+ years software engineering, ML engineering or production AI engineering experience, with evidence of shipping reliable services beyond prototypes.
• 2+ years working with LLMs, including prompt and context engineering, structured outputs, function calling, API integration, RAG, evaluation and production monitoring.
• Experience with agent frameworks and runtimes such as LangGraph, LangChain, LlamaIndex, Semantic Kernel, AutoGen, CrewAI, OpenAI Agents SDK or comparable custom orchestration.
• Strong Python engineering skills including async programming, type hints, Pydantic-style validation, API design, test automation, secure coding and maintainable service architecture.
• Deep understanding of agentic patterns: ReAct, plan-and-execute, reflection, state graphs, multi-agent orchestration, memory design, human-in-the-loop controls and escalation paths.
• Knowledge of retrieval and context systems: embeddings, vector databases, hybrid search, metadata filtering, re-ranking, access-controlled RAG, freshness, citation/provenance and context budgeting.
• Experience designing agent tools and integrations using OpenAPI/function schemas, MCP servers, secure tool wrappers, least-privilege permissions, rate limits, idempotency and audit logging.
• Competence in agent evaluation and observability including golden datasets, adversarial tests, LLM-as-judge with human validation, trace viewers, dashboards, regression gates and production feedback loops.
• Security, privacy and governance knowledge including prompt injection, insecure retrieval, data exfiltration, PII handling, sandboxing, policy enforcement, approvals and compliance-ready audit evidence.
• Tools and technologies: Azure OpenAI, Anthropic Claude, Mistral/Llama models, Hugging Face, Azure AI Search, Pinecone, Weaviate, FAISS, LangSmith, Braintrust, Application Insights, Docker, Kubernetes, GitHub Actions or Azure DevOps.
• Nice-to-haves: MCP server authoring, Agent-to-Agent communication protocols, multimodal agents, Arabic/UAE localisation, regulated-industry AI governance, on-prem/local inference with vLLM or TensorRT-LLM, and model routing for cost/performance optimisation.