[Paper Review] Gradio: Hassle-Free Sharing and Testing of ML Models in the Wild

Abubakar Abid, Ali Abdalla+4 more|arXiv (Cornell University)|Jun 6, 2019

Scientific Computing and Data Management8 references113 citations

TL;DR

Gradio is an open-source Python package that lets researchers quickly wrap ML models with visual web interfaces and share them via public links for interactive testing and feedback in real-world settings.

ABSTRACT

Accessibility is a major challenge of machine learning (ML). Typical ML models are built by specialists and require specialized hardware/software as well as ML experience to validate. This makes it challenging for non-technical collaborators and endpoint users (e.g. physicians) to easily provide feedback on model development and to gain trust in ML. The accessibility challenge also makes collaboration more difficult and limits the ML researcher's exposure to realistic data and scenarios that occur in the wild. To improve accessibility and facilitate collaboration, we developed an open-source Python package, Gradio, which allows researchers to rapidly generate a visual interface for their ML models. Gradio makes accessing any ML model as easy as sharing a URL. Our development of Gradio is informed by interviews with a number of machine learning researchers who participate in interdisciplinary collaborations. Their feedback identified that Gradio should support a variety of interfaces and frameworks, allow for easy sharing of the interface, allow for input manipulation and interactive inference by the domain expert, as well as allow embedding the interface in iPython notebooks. We developed these features and carried out a case study to understand Gradio's usefulness and usability in the setting of a machine learning collaboration between a researcher and a cardiologist.

Motivation & Objective

• Identify barriers to accessibility and collaboration in ML model development with domain experts.
• Develop an open-source tool that simplifies creating interactive interfaces for various model types and data modalities.
• Facilitate secure sharing and real-time feedback from non-technical collaborators.
• Evaluate usability and usefulness through a pilot study in a cardiology collaboration.

Proposed method

• Implement Gradio as a Python library with interfaces for image, text, and audio inputs/outputs.
• Provide an Interface object that specifies input/output types, model type, and the model, with optional pre/post-processing.
• Enable launching in-browser, inline in notebooks, and optional public share links via an SSH-tunneled connection.
• Allow input manipulation (e.g., cropping, occluding) and secure data transmission to the host for inference.
• Capture feedback through a flagging mechanism and messages sent from collaborators to the model host.

Experimental results

Research questions

• RQ1 How do researchers share data and models with and without Gradio?
• RQ2 What features of Gradio are most used and most unused by developers and collaborators?
• RQ3 What kind of model feedback do collaborators provide through the Gradio interface?
• RQ4 What additional features do developers and collaborators request for Gradio?

Key findings

• Gradio enables collaborators to interact with models via a shareable link and client-side browser interaction without installing software.
• In the pilot, the cardiologist could test robustness by occluding parts of ultrasound images and noting model responses.
• The setup and deployment of Gradio took about 10 minutes in the cited workflow, after which collaborators could test models in real time.
• Collaborators frequently used input manipulation and flagging of incorrect outputs to provide targeted feedback.
• Users rarely leveraged the built-in iPython notebook embedding or the explicit share button, preferring direct URL sharing.