Google AI Machine Learning Model Transparency Reporting
Model Card Toolkit for Easier Model Transparency Reporting
The 29th of July there was a post by Huanming Fang and Hui Miao, Software Engineers, at Google Research — on machine learning transparency. I read this first on the Google AI Blog and explored a previous paper on arXiv published January 2019. There is additionally a wonderful illustrated page about this from Google here:
It is really fascinating to think about how a model can be understand better and more equitable by a multitude of stakeholders.
In practice this is what it could look like:
Machine learning (ML) model transparency is important.
The applications that integrate ML are now of such a wide variety.
What information is needed?
What details is important for developers to decide whether a model is appropriate for their use case?
Most models have limitations and can to a greater degree be informed by ethical considerations.
This can benefit developers, regulators, and downstream users alike.
According to the blog there is a project worth checking out:
“For example, the MediaPipe team creates state-of-the-art computer vision models for a number of common tasks, and has included Model Cards for each of their open-source models in their GitHub repository.”
Their repository is really interesting! You can check out the model cards for a variety of models.
Compiling and presenting the information in a format that’s accessible and understandable can be an important task.
To streamline the creation of Model Cards for all ML practitioners, they are sharing the Model Card Toolkit (MCT).
It is a collection of tools that support developers in compiling the information that goes into a Model Card and that aid in the creation of interfaces that will be useful for different audiences.
I’ve taken this information from the GitHub page linked above:
“The Model Card Toolkit (MCT) streamlines and automates generation of Model Cards [1], machine learning documents that provide context and transparency into a model’s development and performance. Integrating the MCT into your ML pipeline enables the sharing model metadata and metrics with researchers, developers, reporters, and more.
Some use cases of model cards include:
- Facilitating the exchange of information between model builders and product developers.
- Informing users of ML models to make better-informed decisions about how to use them (or how not to use them).
- Providing model information required for effective public oversight and accountability.”
What is great in. this regard is that they have also released a Colab tutorial that builds a Model Card for a simple classification model trained on the UCI Census Income dataset.
If you leverage these solutions you can even contact Google at:
model-cards@google.com.
You can learn more about Google’s efforts to promote responsible AI in the TensorFlow ecosystem on their TensorFlow Responsible AI page.
This is #500daysofAI and you are reading article 424. I am writing one new article about or related to artificial intelligence every day for 500 days.
