Undelucram.ro on behalf of: Crowdstrike SRLAbout the Role:The CrowdStrike Data Science Team is looking for an experienced and driven leader to build and guide a team dedicated to designing and building evaluations for AI models that perform cybersecurity tasks. Our mission is to establish rigorous, reproducible standards for measuring how well AI and agentic systems support real-world security operations.As the Evaluation Lead, you will bring deep, hands-on SOC expertise to the table, defining what "good" looks like for AI models operating in security workflows. You will build the datasets and methodologies that ground our models in the realities of frontline defense, and lead a team that turns findings into actionable insights for engineering and product stakeholders.What You'll Do:Lead, mentor and grow a team of researchers and engineers focused on evaluating AI models applied to cybersecurity tasksDefine the strategy, roadmap and success metrics for evaluating AI/LLM and agentic systems across security use cases (e.g. incident response, threat hunting, alert triage, investigation)Design and standardize evaluation methodologies, benchmark datasets and reproducible testing pipelines rooted in real SOC workflowsAssess models for accuracy, robustness, reliability and operational effectiveness in security-analyst scenariosEstablish quantitative and qualitative metrics that reflect how AI models and agentic systems perform against genuine threat-detection and response challengesCollaborate cross-functionally with engineering, product and threat-research teams to translate evaluation findings into scalable improvementsCommunicate results, trade-offs and recommendations clearly to both technical and executive audiencesWhat You'll Need:Hands-on SOC experience with a strong understanding of day-to-day security operationsAt least 3 years in a management or team-leadership position, with a proven track record of mentoring and growing technical teams (formal manager experience is a plus)Strong knowledge of incident response and threat hunting, including detection, investigation and remediation workflowsBroad knowledge of the cybersecurity landscape, including attack vectors, defense mechanisms, and the analyst workflows that AI aims to augmentPractical experience working with AI, including familiarity with AI/LLM capabilities and knowledge of agentic systems and how they operateAbility to define, design and standardize evaluation methodologies and reproducible testing pipelinesExceptional communication skills, including the ability to present complex technical findings clearly to both technical and non-technical audiencesDemonstrated track record of delivering results, supported by shareable projects or measurable outcomesA genuine passion for working at the forefront of AI and cybersecurity, with a proactive mindset toward continuous learning and innovation in a rapidly evolving fieldThe following is required within every job description: "Proven experience utilizing AI technologies to enhance decision-making, streamline workflows and processes, improve efficiency and drive business outcomes.Bonus Points:Relevant security certifications (e.g., GCIA, GCIH, GCFA, OSCP, or equivalent)Experience building or working with LLM/agentic evaluation frameworks and benchmarking pipelinesProgramming experience in Python and familiarity with data-science toolingFamiliarity with SOC platforms, SIEM/SOAR tools, and detection engineeringExposure to adversarial testing, red-teaming, or robustness evaluation of AI systemsUnderstanding of MITRE ATT&CK and how it maps to detection and response workflowsWhat We Offer:Competitive salaryStock optionsPrivate Healthcare insuranceLife insuranceTraining budgetWorking with the latest technologiesFlexible time offTeam hangouts Show more Show less