Awesome Deep Learning TensorFlow Lite – Massive Collection of Resources

This is an awesome list of TensorFlow Lite models with sample apps, helpful tools and learning resources –

• Showcase what the community has built with TensorFlow Lite
• Put all the samples side-by-side for easy reference
• Share knowledge and learning resources

Contents

• What is new
• Models with samples
  • Computer vision
  • Text
  • Speech
• Model zoo
  • TensorFlow Lite models
  • TensorFlow models
• Ideas and Inspiration
• ML Kit examples
• Plugins and SDKs
• Helpful links
• Learning resources
  • Blog posts
  • Books
  • Videos
  • Podcasts
  • MOOCs

What is new

Here are the new features and tools of TensorFlow Lite:

• Announcement of the new converter – MLIR-based and enables conversion of new classes of models such as Mask R-CNN and Mobile BERT etc., supports functional control flow and better error handling during conversion. Enabled by default in the nightly builds.
• Android Support Library – Makes mobile development easier (Android sample code).
• Model Maker – Create your custom image & text classification models easily in a few lines of code. See below the Icon Classifier for a tutorial by the community.
• On-device training – It is finally here! Currently limited to transfer learning for image classification only but it’s a great start. See the official Android sample code and another one from the community (Blog | Android).
• Hexagon delegate – How to use the Hexagon Delegate to speed up model inference on mobile and edge devices. Also see blog post Accelerating TensorFlow Lite on Qualcomm Hexagon DSPs.
• Model Metadata – Provides a standard for model descriptions which also enables Code Gen and Android Studio ML Model Binding.

Models with samples

Here are the TensorFlow Lite models with app / device implementations, and references.
Note: pretrained TensorFlow Lite models from MediaPipe are included, which you can implement with or without MediaPipe.

Computer vision

Text

Speech

Model zoo

TensorFlow Lite models

These are the TensorFlow Lite models that could be implemented in apps and things:

• MobileNet – Pretrained MobileNet v2 and v3 models.
• TensorFlow Lite models
  • TensorFlow Lite models – With official Android and iOS examples.
  • Pretrained models – Quantized and floating point variants.
  • TensorFlow Hub – Set “Model format = TFLite” to find TensorFlow Lite models.

TensorFlow models

These are TensorFlow models that could be converted to .tflite and then implemented in apps and things:

• TensorFlow models – Official TensorFlow models.
• Tensorflow detection model zoo – Pre-trained on COCO, KITTI, AVA v2.1, iNaturalist Species datasets.

Ideas and Inspiration

• E2E TFLite Tutorials – Checkout this repo for sample app ideas and seeking help for your tutorial projects. Once a project gets completed, the links of the TensorFlow Lite model(s), sample code and tutorial will be added to this awesome list.

ML Kit examples

ML Kit is a mobile SDK that brings Google’s ML expertise to mobile developers.

• 2019-10-01 ML Kit Translate demo – A tutorial with material design Android (Kotlin) sample – recognize, identify Language and translate text from live camera with ML Kit for Firebase.
• 2019-03-13 Computer Vision with ML Kit – Flutter In Focus.
• 2019-02-09 Flutter + MLKit: Business Card Mail Extractor – A blog post with a Flutter sample code.
• 2019-02-08 From TensorFlow to ML Kit: Power your Android application with machine learning – A talk with Android (Kotlin) sample code.
• 2018-08-07 Building a Custom Machine Learning Model on Android with TensorFlow Lite.
• 2018-07-20 ML Kit and Face Detection in Flutter.
• 2018-07-27 ML Kit on Android 4: Landmark Detection.
• 2018-07-28 ML Kit on Android 3: Barcode Scanning.
• 2018-05-31 ML Kit on Android 2: Face Detection.
• 2018-05-22 ML Kit on Android 1: Intro.

Plugins and SDKs

• Edge Impulse – Created by @EdgeImpulse to help you to train TensorFlow Lite models for embedded devices in the cloud.
• Fritz.ai – An ML platform by @fritzlabs that makes mobile development easier: with pre-trained ML models and end-to-end platform for building and deploying custom trained models.
• MediaPipe – A cross platform (mobile, desktop and Edge TPUs) AI pipeline by Google AI. (PM Ming Yong) | MediaPipe examples.
• Coral Edge TPU – Edge hardware by Google. Coral Edge TPU examples.
• TensorFlow Lite Flutter Plugin – Provides a dart API similar to the TensorFlow Lite Java API for accessing TensorFlow Lite interpreter and performing inference in flutter apps. tflite_flutter on pub.dev.

Helpful links

• Netron – A tool for visualizing models.
• AI benchmark – A website for benchmarking computer vision models on smartphones.
• Performance measurement – How to measure model performance on Android and iOS.
• Material design guidelines for ML – How to design machine learning powered features. A good example: ML Kit Showcase App.
• The People + AI Guide book – Learn how to design human-centered AI products.
• Adventures in TensorFlow Lite – A repository showing non-trivial conversion processes and general explorations in TensorFlow Lite.
• TFProfiler – An Android-based app to profile TensorFlow Lite models and measure its performance on smartphone.
• TensorFlow Lite for Microcontrollers
• TensorFlow Lite Examples – Android – A repository refactors and rewrites all the TensorFlow Lite Android examples which are included in the TensorFlow official website.

Learning resources

Interested but not sure how to get started? Here are some learning resources that will help you whether you are a beginner or a practitioner in the field for a while.

Blog posts

• 2020-04-20 What is new in TensorFlow Lite – By Khanh LeViet.
• 2020-04-17 Optimizing style transfer to run on mobile with TFLite – By Khanh LeViet and Luiz Gustavo Martins.
• 2020-04-14 How TensorFlow Lite helps you from prototype to product – By Khanh LeViet.
• 2019-11-08 Getting Started with ML on MCUs with TensorFlow – By Brandon Satrom.
• 2019-08-05 TensorFlow Model Optimization Toolkit — float16 quantization halves model size – By the TensorFlow team.
• 2018-07-13 Training and serving a real-time mobile object detector in 30 minutes with Cloud TPUs – By Sara Robinson, Aakanksha Chowdhery, and Jonathan Huang.
• 2018-06-11 – Why the Future of Machine Learning is Tiny – By Pete Warden.
• 2018-03-30 – Using TensorFlow Lite on Android) – By Laurence Moroney.

Books

• 2020-03-01 Raspberry Pi for Computer Vision (Complete Bundle | TOC) – By the PyImageSearch Team: Adrian Rosebrock (@PyImageSearch), David Hoffman, Asbhishek Thanki, Sayak Paul (@RisingSayak), and David Mcduffee.
• 2019-12-01 TinyML – By Pete Warden (@petewarden) and Daniel Situnayake (@dansitu).
• 2019-10-01 Practical Deep Learning for Cloud, Mobile, and Edge – By Anirudh Koul (@AnirudhKoul), Siddha Ganju (@SiddhaGanju), and Meher Kasam (@MeherKasam).

Videos

• 2020-07-25 Android ML by Hoi Lam (GDG Kolkata meetup).
• 2020-04-01 Easy on-device ML from prototype to production (TF Dev Summit 2020).
• 2020-03-11 TensorFlow Lite: ML for mobile and IoT devices (TF Dev Summit 2020).
• 2019-10-31 Keynote – TensorFlow Lite: ML for mobile and IoT devices.
• 2019-10-31 TensorFlow Lite: Solution for running ML on-device.
• 2019-10-31 TensorFlow model optimization: Quantization and pruning.
• 2019-10-29 Inside TensorFlow: TensorFlow Lite.
• 2018-04-18 TensorFlow Lite for Android (Coding TensorFlow).

Podcasts

• 2020-08-08 Talking Machine Learning with Hoi Lam.

MOOCs

• Introduction to TensorFlow Lite – Udacity course by Daniel Situnayake (@dansitu), Paige Bailey (@DynamicWebPaige), and Juan Delgado.
• Device-based Models with TensorFlow Lite – Coursera course by Laurence Moroney (@lmoroney).
• The Future of ML is Tiny and Bright – A series of edX courses created by Harvard in collaboration with Google. Instructors – Vijay Janapa Reddi, Laurence Moroney, and Pete Warden.