Agile for AI: Managing the Uncertainty of Machine Learning Projects
Machine learning projects defy traditional planning because outcomes are fundamentally unpredictable until the data speaks. Discover how Agile's empirical principles map onto the unique uncertainty of AI development — and where they need to be adapted.
The Planning Problem in ML
Software engineers building a CRUD application can reasonably estimate how long a feature will take. They've built similar features before. The complexity is knowable. The outcome, while uncertain, is bounded.
Machine learning engineers building a fraud detection model cannot tell you with confidence whether their model will achieve 95% precision or 72% — or whether it will even be deployable on the target hardware within the required latency budget. The outcome depends on data quality, feature engineering choices, algorithm selection, hyperparameter tuning, and the often-surprising characteristics of real-world production data that no amount of upfront planning can fully anticipate.
This fundamental uncertainty makes traditional project management frameworks nearly useless for ML work — and it makes Agile, done right, an almost natural fit. Almost. Because even Agile requires some adaptation to work well in ML contexts.
Where Agile Fits ML Development Naturally
Iterative Experimentation
The ML development cycle — hypothesize, experiment, evaluate, iterate — maps directly onto Agile's inspect-and-adapt loop. Each model training run is effectively a sprint: a time-boxed effort to move toward a goal, producing an artifact that can be evaluated against acceptance criteria.
Treating model development as a sequence of experiments rather than a linear progression from requirements to delivery normalizes the unpredictability that plagues traditionally managed AI projects. A sprint that produces a model with 65% accuracy when you expected 85% is not a failed sprint — it's a learning sprint that should directly inform the next iteration's hypothesis.
Customer Collaboration on Criteria
One of the most valuable Agile practices in ML contexts is the explicit, upfront alignment between data scientists and business stakeholders on what "good enough" looks like — before a single model is trained. What accuracy threshold makes this model worth deploying? What false positive rate is tolerable given the business cost of each false positive? What latency is acceptable in production?
These questions have answers that business stakeholders can provide, but they're rarely asked systematically in traditionally managed ML projects. Agile's emphasis on continuous stakeholder collaboration creates natural forcing functions for these conversations.
Working Software Over Documentation
In ML, the equivalent of "working software" is a validated model artifact: a model that performs adequately on a held-out test set, whose behavior is understood by the team, and that is deployable to a production or staging environment. Orienting sprints around this deliverable — rather than around intermediate artifacts like data pipelines or feature engineering code — keeps teams focused on customer-relevant outcomes.
Where Standard Agile Needs Adaptation
The Sprint Goal Challenge
Scrum's sprint goal model assumes that teams can commit at the start of the sprint to delivering a specific, defined increment. In ML, this commitment is often impossible: you can commit to running a specific set of experiments, but you cannot commit to achieving a specific model performance outcome.
This requires a reframing of ML sprint goals: not "achieve 90% recall on the fraud model" but "complete experiments with three feature engineering approaches and document results." The goal is process and learning, not predetermined outcome.
The Definition of Done
Traditional Definition of Done criteria (code reviewed, tests passing, deployed to staging) need ML-specific additions:
- Model performance meets defined thresholds on test dataset - Model behavior on edge cases is documented and reviewed - Bias and fairness evaluation is complete - Model explainability documentation exists (for regulated use cases) - Monitoring alerts for distribution shift are configured - Rollback procedure is tested
Without ML-specific DoD criteria, "done" becomes a dangerously ambiguous concept in AI projects.
Managing Research vs. Delivery Sprints
Not all ML work is delivery work. Some sprints are fundamentally exploratory — literature review, dataset analysis, proof-of-concept experimentation. This research work doesn't produce a deployable increment, but it produces knowledge that makes future delivery sprints more effective.
Mature ML teams distinguish explicitly between research sprints (output: documented learnings) and delivery sprints (output: deployable model artifact) and hold them to different standards of success. Blurring the two creates unrealistic expectations and misleading velocity metrics.
Practical Patterns for Agile ML Teams
**Model versioning as source control.** Treat model artifacts the way software teams treat code: version controlled, reproducible from a defined set of inputs, with a clear history of what changed between versions and why.
**Experiment tracking from day one.** Tools like MLflow, Weights & Biases, or Neptune give teams a systematic record of experiments — hyperparameters, training data versions, evaluation metrics. Without this, teams lose institutional memory of what was tried, what worked, and what didn't.
**Separate ML pipelines from application code.** Data ingestion, feature engineering, model training, and model serving are distinct concerns that benefit from separate codebases, separate CI/CD pipelines, and potentially separate team ownership. Conflating them creates fragile systems and unclear ownership.
**Regular model performance reviews.** Models degrade as production data distributions shift away from training data. Agile ML teams treat model performance monitoring as an ongoing sprint activity, not a post-deployment afterthought — and have a clear protocol for when degrading performance triggers a retraining sprint.
The organizations that manage AI development well treat it as a fundamentally empirical discipline — closer to scientific research than traditional software engineering — and apply Agile's inspect-and-adapt mechanisms accordingly. The teams that struggle treat ML like feature development with longer timelines, and discover too late that the planning assumptions don't hold.
Ready to put this into practice?
Join a SAFe certification course and master agile at scale.
Browse Courses →
