Camera Image Processing on ESP32 Microcontroller with Help of Convolutional Neural Network
DOI:
https://doi.org/10.18372/1990-5548.72.16939Keywords:
transfer learning, microcontrollers, image classification, ESP32Abstract
This paper analyzes a common ESP32 microcontroller with a built-in camera for image classification tasks using a convolutional neural network. ESP32 is commonly used in IoT devices to read data and control sensors, so its computing power is not significant, which has a positive effect on the cost of the device. The prevalence of ultra-low power embedded devices such as ESP32 will allow the widespread use of artificial intelligence built-in IoT devices. The duration of photographing and photo processing is obtained in the paper, as this can be a bottleneck of the microcontroller, especially together with machine learning algorithms. Deployed convolutional neural network, pre-trained on another device, MobileNet architecture on microcontroller and proved that ESP32 capacity is sufficient for simultaneous operation of both the camera and convolutional neural network.
References
D. Schweizer, M. Zehnder, H. Wache, H. Witschel, D. Zanatta and M. Rodriguez. Using Consumer Behavior Data to Reduce Energy Consumption in Smart Homes: Applying Machine Learning to Save Energy without Lowering Comfort of Inhabitants, 2015. https://doi.org/10.1109/ICMLA.2015.62
Liciotti, Daniele & Bernardini, Michele & Romeo, Luca & Frontoni, Emanuele. A Sequential Deep Learning Application for Recognising Human Activities in Smart Homes, Neurocomputing, 2019, pp. 396. https://doi.org/10.1016/j.neucom.2018.10.104
Keras documentation: MobileNet, MobileNetV2, and MobileNetV3. Keras: the Python deep learning API. https://keras.io/api/applications/mobilenet/
Jian Mao, Qixiao Lin, Jingdong Bian. Application of learning algorithms in smart home IoT system security, 2018. https://doi.org/10.3934/mfc.2018004
L. Y. Pratt, Discriminability-based transfer between neural networks. NIPS Conference: Advances in Neural Information Processing Systems, 1992.
STM32Cube.AI: Convert neural networks into optimized code for STM32. ST life.augmented Blog. https://blog.st.com/stm32cubeai-neural-networks/
L. Lai,, N. Suda, I. V. Chandra, CMSIS-NN: efficient neural network kernels for arm cortex-M CPUs. Comput. https://arxiv.org/abs/1801.06601, 2018
General vision, Presentation of the Curie Neurons on Arduino/Genuino101,https://www.general-vision.com/publications/PR_CurieNeuronsPresentation.pdf
Allan, A. Getting started with the NVIDIA jetson nano developer kit, 2019
M5-docs. https://docs.m5stack.com/en/unit/m5camera
Pseudostatic (random-access) memory (PSRAM) | JEDEC. https://www.jedec.org/standards-documents/dictionary/terms/pseudostatic-random-access-memory-psram, 2019
Micros() – arduino reference. Arduino - Home. https://www.arduino.cc/reference/en/language/functions/time/micros/
Stephen Johnson. Stephen Johnson on Digital Photography. O'Reilly. ISBN 0-596-52370-X , 2006
Contributors to Wikimedia projects. ImageNet - Wikipedia. Wikipedia, the free encyclopedia. https://en.wikipedia.org/wiki/ImageNet
Intro to machine learning with edge impulse - silicon labs. (2020). Silicon Labs. https://www.silabs.com/support/training/intro-machine-learning-with-edge-impulse/intro-machine-learning-with-edge-impulse-presentation
Tensorflow.(2022). models/research/slim/nets/ mobilenet at master tensorflow/models. GitHub.(2022),https://github.com/tensorflow/models/tree/master/research/slim/nets/mobilenet
NetAdapt: Platform-aware neural network adaptation for mobile applications. (2018). arXiv.org. https://arxiv.org/abs/1804.03230
Downloads
Published
Issue
Section
License
Authors who publish with this journal agree to the following terms:
Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
