Overview / Usage

Demo:

• Blog: http://fungai.org
• ReactJS UI POC: https://fungai-react-ui.herokuapp.com/fungpredict
• Fake json-server POC: https://fungai-json-server-heroku.herokuapp.com/

Fungi Barbarian is a wild mushroom ID machine. It's goal is to assist a human identifying wild mushrooms: given some sample wild mushroom images, Fungi Barbarian will try and identify the mushroom for you. It does not require internet access and is able to perform the inferences offline. This project is originally motivated and inspired by a friends Facebook post: "hey do you know what this mushroom is?".

Core Technology Stack (for offline training and inference):

• Model Training: Intel Xeon Phi Cluster: to train the image classification model - leveraging Transfer Learning on Google Inception V3 model.
• Prediction / Inference: Movidius Neural Compute Stick (NCS), loaded with a pre-trained Tensorflow image classification model specialises in wild mushroom images
• Hardware: Raspberry Pi (e.g. MyCroft AI), or whatever suitable that is able to link up camera (for image capture), NCS, and portable power via USB connection. The entire inference solution is offline and does not require internet access.

Secondary Technology Stack (for online inference):

• Full-stack web application to enable inferences via a web application (in case user prefer to do inferences in batches after their wild-mushroom ID trips). Frontend: ReactJS single page web app. API calls with backends (e.g. tensorflow model exposed via Tensorflow, ImageNet image URLs to grab images for training, validation, and testing (inferences),

Training Data Repository:

• ImageNet: http://image-net.org/download-imageurls
• Wikimedia: to store training data (images), with free / open / permissive licence.
• or whatever repositories suitable for storing open data in suitable format (e.g. images, video, sensor data, etc.)

Documentations

• Blog Posts
• Interactive full-stack web application for educational purposes.

Vision

• Free everything: free data (Wikimedia?), free software (GitHub repository), free instructions (via documentations). It is important anyone can replicate / create their own solution / be inspired from this project, without the hindrance of copy right issues. The licence for this project will be highly permissible (e.g. MIT).
• Collaborate with local wild mushroom ID (human) experts on building up an reliable / accurate / trustworthy data repository on Wild Mushroom. Data should contains features (e.g. images, videos, sensor data, etc.) and labels (annotated / identified by a human expert). i.e. Whatever it is required for Supervised Training in Machine Learning terms.
• sustainable ecosystem with strong governance (something to think about).
• potential ongoing sponsorship opportunities from local nature reserve / parks / communities (?)

Foreseeable challenges and strategies:

• As there are literally thousands (if not more) species of wild mushrooms out there, it could be a challenge to collect sufficient training data. To make this project practical, it may be wise to start off with just a small group of well known categories (e.g ImageNet categories: "Earthstar", "Agaric", etc. ) - say pick 10 categories. Train the model very well against these specialised categories. Only when we have a reasonable good proof of concept we then move on to repeat process for other mushroom categories / species.
• Most images found on the web have copyright protection on them, and hence may not be suitable for consumption for this project. To work around this, try focuses on free and open images only (e.g. ImageNet). If we are going to generate our own data, make sure we keep our data free and open - to avoid copyright hindrance.
• For the purpose of initial proof of concept, it might be wise to pick a ImageNet category at an appropriate level to ensure images are sufficient "different" to the human eye for good demo purposes (The sub categories underneath the Agaric category is pretty good for this). In future if we wish to go deeper (e.g. to differentiate the different types of Earthstars), we may need to perform a more specialized form of transfer learning - to make the differentiation more sensitive. Think CT Scan images but Fungus images.
• ImageNet ImageNet ImageNet.