Use Custom Attributes to Capture Model Metadata to Manage & Track Your Models

If your Governance team asked you for a list of all your models or model versions that are subject to compliance with rules around personally identifiable information (PII), that incorporate generative AI, or that are classified as "high risk" under the EU AI Act, how long would it take you and your team to assemble that list?

What if your colleague Jane left the company to take a data scientist position elsewhere, and you needed to review all the models that she managed? Or you needed to report on all the models being consumed by the Android Application Team?

Documentation plays a crucial role in the successful implementation and maintenance of models. However, the documentation process for these models is often ad hoc and decentralized, leading to various challenges and inefficiencies:

• Different teams use different taxonomies to describe model attributes, making it difficult for team members or stakeholders to understand and use the models effectively.
• Inconsistent documentation practices contribute to knowledge silos within organizations. When team members change roles or leave the organization, critical knowledge may be lost.
• Poor documentation makes collaboration between data scientists, engineers and other stakeholders challenging and limits the ability to iterate and improve models effectively.
• Inadequate documentation can pose significant governance and compliance risks, as it becomes difficult to track which models are subject to which regulatory requirements. 

How Custom Attributes Can Help

As part of a robust model catalog, the ability to set custom attributes for models and model versions offers a powerful solution to the documentation challenges faced by AI/ML teams. 

With all of an organization's models registered in the catalog, and with a standard set of custom attributes defined for the organization, ML teams can ensure that they are able to:

• Define and capture essential information consistently across models using a standardized documentation framework. 
• Centralize and organize documentation efforts, fostering knowledge sharing, mitigating knowledge loss risks, and promoting cross-team collaboration.
• Facilitate reproducibility by capturing critical model information that allows for easier replication of experiments, faster troubleshooting and more efficient model iteration.
• Capture compliance-related information to meet governance requirements, demonstrate model accountability, and ensure adherence to ethical guidelines, reducing compliance risks.

Using Custom Attributes in Verta Model Catalog

Verta Model Catalog lets you create and set custom attributes to capture metadata for all your models and model versions. For example:

• Create a custom attribute called "Owner" to denote which team owns a given model.
• Capture which business unit is consuming the results of a model or endpoint.
• Or score your models based on a custom risk hierarchy.

Whatever attribute is important to you, your team or your organization, you can easily capture that information for each model or version. Additionally, you can attach arbitrary key-value pairs to a registered model version via our Python client.

How to Use Custom Attributes

Using custom attributes requires two steps. First, an organization admin within the Verta platform needs to name and configure the attribute. Secondly, the attribute can be populated on your objects.

Teams can set up an unlimited number of custom attributes. The attributes can be set up to allow choosing from a list of users or groups (e.g., when you want to designate the model owner or consuming team), or you can create your own list of values for users to choose from.

Once a custom attribute is created, the list of all your attributes will be visible on the summary page of a registered model or version, and users can set any or all of the attributes for the models that they have access to.

In summary, embracing custom attributes can empower organizations to harness the full potential of their models while promoting efficiency, governance, collaboration and long-term success.

Learn more about using custom attributes in Verta Model Catalog with this 2-minute video introduction.