Today’s data scientists are often asked “How can you ship models to production faster without breaking things?”
Over the past few years, machine learning has seen exponential growth in enterprises. The emergence of tools that automate the model development and training workflows has helped fast-track model development. By the year 2024, 75% of the companies are planning to graduate from pilot to operations; however, as per Andrew Ng “We are still struggling to take promising POCs and turn them into practical production deployments”.
The model registry is a system that allows data scientists to publish their production-ready models and share them for collaboration with other teams and stakeholders. MLOps cannot be done right until you have a state-of-the-art Model Registry.
Here are the top 3 benefits that a Model Registry can provide to you.
The ad hoc processes make it harder to identify which models are production-ready. Typically you are not comfortable sharing the tens and thousands of iterations with your entire team but rather the best fit model.
Once a winning model is chosen, the handoff between data science and operations is not an elegant one. The norm is that data scientists throw a model over the fence to engineers who try to fit ML into the software release processes. An amazingly running model on your laptop should be guaranteed to deliver similar predictions when engineers deploy in staging or prod. Does that ever happen? The lack of ability to track the four key model ingredients - code, data, configuration (hyperparameter), and the computing environment for every model introduces inconsistencies and delays.
Machine learning models need to constantly adapt to change in market conditions, data, and concept drifts. Hence, data science teams need to react quickly to train and deploy new model versions and at Verta, we have worked with customer teams releasing new model versions almost daily. A manual process of packaging and promoting a model to production cannot scale and the release pipeline needs to be fully automated.
This is where the model registry comes in. After the experimentation phase when you are ready for deployment data scientists can select the best fit models from all the experiments and stage them for release in the registry.
Similar to Dockerhub or Artifactory from the software universe, model registry guarantees a flexible and reliable model release process by allowing data scientists to build/publish models for release along with all the model metadata and artifacts in one central repository. Model Registries provide interoperability with any model type, regardless of where models are pushed or pulled from, eliminating model performance inconsistencies and delay in the roll-out.
ML Models are used by applications to make important decisions like preventing fraud, approving credits, detecting diseases, etc. These applications need validation on different aspects like regulatory compliance, data privacy, security, and bias. However, the lack of security and governance in model development today is stunning.
Data privacy concerns like using PII data in training models without anonymization cannot be an afterthought. It is probably not difficult for hackers to steal sensitive information from Machine Learning classifiers. Similarly, if you get audited for GDPR compliance, you will be needed to cleanse your database of any un-opted-in privacy data. This may mean a total loss of your training data set.
ML teams are far behind on the adoption of vulnerability scanning for model code. The ability to track all the underlying ML libraries e.g. NumPy/SciPy and look for vulnerabilities is mostly missing in the release process. Instead, we need the right tools to guarantee security.
A model registry can help implement a formal model governance and approval process from legal, business, or technical stakeholders prior to deployment. You can create a company-wide model inventory that lists all the ML models, associated data, their usage, interdependencies, and assigned risk levels.
A central source of truth for models across different stages of their lifecycle from development, staging, and production can help deploy and scale machine learning projects reliably and ensure governance for all model assets across your organization and infrastructure.
The head of data science from a large US bank once told us ”We have half a dozen teams using half a dozen different ways of building models..as many places to run them...we have no central place to get visibility at an organizational level.” Organizations with hundreds of models today face the challenge arising from the zoo of ML workflows and data scientists being put into silos within the nature of large enterprises.
With silos throughout the company, it’s hard for a data scientist to discover what models have already been built and refined by other teams. Everyone ends up reinventing the wheel or you cannot take someone’s battle-tested model and build on it. This is where a model registry can enable and empower a data science practice. It makes it easy to share models and serves as a sort of data warehouse where models can be discovered, curated, or collaborated upon.
With a state-of-the-art model registry, you have a central source of truth for your models across different stages of their lifecycle, including development, validation, deployment, and monitoring. You will create better models together and put them to use faster and with confidence!
At Verta (creators of ModelDB), we provide a “Model Registry” with features like model registration, version control, artifact management, model annotation & documentation, model lifecycle stage transition, risk tagging, and approval workflows.