If you’ve been around software-enabled businesses for any length of time, you’re probably already familiar with Agile practices. And with good reason. Agile allows teams to work quickly and collaboratively to deliver value with fewer headaches — even in uncertain, turbulent environments. Compared to a traditional waterfall model of software development, Agile allows teams to test hypotheses, get feedback and decide on scope and direction at a faster pace. As a result, Agile practices have become the de facto way to get things done for most tech-enabled ventures.
In our opinion, the principles and practices behind Agile — as outlined in the Manifesto for Agile Software Development — make Agile a great way to work on AI and ML initiatives.
For the reasons described above, Agile is a great fit for the projects that Machine Learning or Data Science teams undertake. But there’s a catch. Incorporating Agile into machine learning is not as easy as simply copy/pasting Agile DevOps processes into an MLOps environment. For a host of reasons, Agile practices that work well for software development don’t always translate into AI or ML initiatives. For example:
The mindset of data scientists is very different from that of software developers. In software, engineers have well-defined acceptance criteria and a very mature set of technologies they are working with, so targets are often well-defined. The same cannot be said for data scientists, as the nature of data science is uncertain and always shifting. Algorithms can change quickly. Infrastructures can shift mid-project as criteria for success evolve. Much of what is going in the field of data science is still taking place in research labs.
As leaders in MLOps, we’ve worked with countless organizations at almost every phase of the model lifecycle — from experimentation to operationalization. In our experience, we believe that AI-enabled organizations should strive to be as Agile as possible. But companies cannot simply expect a software-based Agile methodology to work well when applied to an AI project. depending on the company and the organization and the type of AI project, processes and methodologies must be refined and adapted for best results. What follows are a few tips for making Agile work for AI:
A Kanban model works for many ML teams since the methodology is fairly lightweight (with few ceremonies) and facilitates more open-ended project planning. Since many ML projects cannot be scoped and defined within a sprint boundary, a Kanban board can help create a list of high-value items that a team can move through (in progress and close states). Kanban also permits teams to continue to scope work as it is in progress.
But Kanban is not a one-size-fits-all solution to Agile for AI and ML initiatives, as the approach contains a few potential drawbacks. Kanban may cause teams to lose sight of the broader picture and business context surrounding a project. And teams using Kanban can sometimes unwittingly work themselves in a silo — finding it difficult to integrate with the rest of the product teams.
Teams should also consider customizing scrum processes to fit into data science projects. Even though some software engineers might believe otherwise, AI teams don’t need to put strict 2-week sprints on development processes. For example, opt for smaller or longer-duration sprints to accommodate your project type and rely on spikes for research-oriented work. This will allow teams to not spend time pondering overestimation or definition of done for open-ended features.
Lastly, we recommend using a hybrid approach. Splitting projects into two distinct phases: model experimentation and model release.
Model experimentation (R&D) involves a lot of open questions and evaluations. This phase tends to be more spike-based, and those sprints often have unpredictable outcomes. We’ve experienced that teams are usually more successful using less rigid Agile frameworks at this phase (discussed in greater detail below).
The closer a team gets to a model release, Agile practices should become more rigid. While getting a model ready for release, teams can spin up formal Scrum (with regular sprints, user stories, etc.) with additional stakeholders. At this phase, you may require integrations with consuming teams, model optimization, testing/validation, etc. to get the model to the last mile, for instance release planning, stakeholder engagement, etc.
In our opinion, AI and ML teams should absolutely strive to become more Agile. But we should focus on embracing the ethos of flexibility and agility behind Agile — not shackling ourselves to processes that might not always fit. Don’t be afraid to experiment and iterate until you find a way of working that works for your team and project.