[Paper Review] TensorFlow with user friendly Graphical Framework for object detection API

Heemoon Yoon, Sanghee Lee+1 more|arXiv (Cornell University)|Jun 11, 2020

Advanced Neural Network Applications4 citations

TL;DR

This paper introduces TensorFlow Graphical Framework (TF-GraF), a user-friendly GUI for TensorFlow's Object Detection API that enables users to perform data preprocessing, model training, evaluation, and inference without coding or command-line interfaces. By providing a visual, interactive environment with support for multiple models (SSD, Faster R-CNN, etc.) and real-time monitoring, TF-GraF lowers the barrier to entry for non-experts in deep learning while managing complex configurations automatically.

ABSTRACT

TensorFlow is an open-source framework for deep learning dataflow and contains application programming interfaces (APIs) of voice analysis, natural language process, and computer vision. Especially, TensorFlow object detection API in computer vision field has been widely applied to technologies of agriculture, engineering, and medicine but barriers to entry of the framework usage is still high through command-line interface (CLI) and code for amateurs and beginners of information technology (IT) field. Therefore, this is aim to develop an user friendly Graphical Framework for object detection API on TensorFlow which is called TensorFlow Graphical Framework (TF-GraF). The TF-GraF provides independent virtual environments according to user accounts in server-side, additionally, execution of data preprocessing, training, and evaluation without CLI in client-side. Furthermore, hyperparameter setting, real-time observation of training process, object visualization of test images, and metrics evaluations of test data can also be operated via TF-GraF. Especially, TF-GraF supports flexible model selection of SSD, Faster-RCNN, RFCN, and Mask-RCNN including convolutional neural networks (inceptions and ResNets) through GUI environment. Consequently, TF-GraF allows anyone, even without any previous knowledge of deep learning frameworks, to design, train and deploy machine intelligence models without coding. Since TF-GraF takes care of setting and configuration, it allows anyone to use deep learning technology for their project without spending time to install complex software and environment.

Motivation & Objective

• To reduce the high barrier to entry for TensorFlow Object Detection API among non-experts and beginners in IT.
• To eliminate reliance on command-line interfaces and manual code configuration for model training and inference.
• To provide a secure, isolated virtual environment per user on the server side for collaborative and scalable usage.
• To support flexible model selection including SSD, Faster R-CNN, RFCN, and Mask R-CNN through a visual interface.
• To enable real-time monitoring of training, visualization of object detection results, and metrics evaluation without coding.

Proposed method

• Designing a web-based graphical interface hosted on a server to manage user accounts and isolated virtual environments.
• Implementing backend services to handle data preprocessing, model training, and evaluation workflows without CLI interaction.
• Integrating TensorFlow Object Detection API with a GUI-driven pipeline for hyperparameter configuration and model selection.
• Enabling real-time visualization of training loss and detection results during model training.
• Supporting multiple deep learning architectures (e.g., Inception, ResNet) via dropdown-based model selection in the GUI.
• Automating environment setup and dependency management to eliminate local installation complexity.

Experimental results

Research questions

• RQ1 How can a graphical interface reduce the learning curve for using TensorFlow’s Object Detection API among non-expert users?
• RQ2 What features are essential in a GUI to support end-to-end object detection workflows without code?
• RQ3 Can a server-side virtual environment model ensure security and scalability for multiple users?
• RQ4 How effective is a GUI-based approach in enabling real-time monitoring and visualization of training and inference processes?
• RQ5 To what extent can a GUI replace command-line workflows in training and evaluating deep learning models for object detection?

Key findings

• TF-GraF successfully enables users to perform data preprocessing, model training, and evaluation entirely through a GUI, eliminating the need for command-line operations.
• The framework supports multiple state-of-the-art models including SSD, Faster R-CNN, RFCN, and Mask R-CNN with configurable hyperparameters via the interface.
• Real-time training monitoring and visualization of detected objects on test images are natively supported in the GUI.
• The system manages complex environment configurations automatically, allowing users to focus on model design and experimentation.
• The codebase for TF-GraF is publicly released, promoting accessibility and community adoption.
• Users without prior deep learning experience can now deploy and experiment with object detection models without writing code.