Everyone Hates Model Documentation. Verta Is Changing That.

It’s no secret that documentation is, well, boring. Writing documentation is the least interesting part of the day for creators like data scientists and software engineers. 

ML model documentation is particularly tedious because you have to document not only the model object itself, but also how the model came to be, its performance and limitations, and explainability and fairness metrics. Little wonder that data scientists report spending 40+ hours to document a single model, according to a recent study.

At the same time, we all know that model documentation is increasingly essential to prevent knowledge loss when data scientists leave the company, ensure that the users of a model have the information they need to use it properly, and comply with increasing AI regulations or prevent massive fines.

And so, at Verta, we want to help model builders write documentation without distracting from creative work!

Seeing Is Believing 

Watch this demo to learn how easy it is to use Verta's AI assistant to quickly create complete and accurate model documentation   

Leveraging AI to Document AI

LLMs are well known to be incredibly effective at writing and summarizing text and code. So we decided to leverage AI to document AI.  

Today we are launching into public preview Verta's brand new AI Assisted Documentation (AIAD) capabilities. AIAD is a “documentation copilot” that handles the heavy lifting of collecting, collating and formatting information required to document a model. You can then review and amend the documentation produced by your AI assistant to ensure that it meets required standards and accurately incorporates all the necessary data.

To get started, just upload the model into the Verta Model Catalog and let your AI sidekick make documentation a breeze.

Verta’s AIAD capabilities combine three simple ideas:

1. Providing templates for model documentation following best practices (e.g., model cards, system cards)
2. Providing LLM-enabled writing assistance and suggestions based on the model
3. Automatically generating documentation using information contained in the model itself (e.g., APIs)

To optimize the LLM outputs for model documentation, we trained the model on a library of well-known model cards to replicate the tone, style, and level of detail. We also spent many cycles optimizing and validating the results for a variety of types of models to minimize hallucinations. 

We've seen different attempts to systematize the requirements for model documentation, such as Google's model cards. But these efforts haven’t gained traction because data scientists simply don’t have the time or writing support to complete them. In fact, a recent CHI paper on documentation found that model cards from sources like Hugging Face and GitHub often were vague, incomplete, and failed to address ethical dimensions of models. Verta's AI-Assisted Documentation turns model cards from great-in-theory to great-in-practice.

Accelerate Documentation 10x Starting Now

Our initial tests suggest that Verta’s AIAD capabilities can accelerate documentation creation by up to 10X, but we think we can push it even further!

Upload your models to see how AIAD can help you write model documentation.

Happy writing, building!