I m now developing a Fashion recommender system that has Four phases
-
Detector the task of the detector to detect objects in the input image and then pass the detected objects to the classification model.
-
Category Classifier the task of the classifier is to classify the image.
-
Attribute Classifier the task of this classifier is to predict the attribute of fashion objects for example clothes have attributes like cotton, wool.
-
Recommender the task of the recommender is to give recommendations based on the selected object in image.
Just like Goggle Lens which detect objects in the image and also gives u ability to select the portion in the image and gives recommendation on it. we have to create this kind of system for Fashion images.
- phase 1:
Detection
the main UI for WebApp would look like this, in this image detector detects objects and mark a box by using OpenCV
I have created a Detector using the Deepfashion2 dataset using YoloV4 and it is showing some good results but data is limited there is no footwear, watch, rings, necklace, sunglasses data in deep fashion, So I manually collected the data by scraping from ALiExpress and using GoogleDataset
But not able to find satisfactory results by adding some additional data. So I decided to go for creating a system first then improve the accuracy of the Detector and other parts of the system.
Here is the result of the detector: Video
- Phase2 and 3:
Classification and Attribute prediction:
I have created two models one for classification and the second for an attribute by using Fastai. These classifiers also need to be more accurate.
- Phase 4:
Recommender System
I am now studying how to create the best recommendation system. i found these methods which show some moderate results.
AutoFaiss
I used autofaiss which creates embeddings and creates an index file. which is way faster than other approaches.
Here is a look at the web app when we select a detected Box it will show us recommendations.
We will create this web app using Django at the backend and Vue.js for the Frontend.
Does anyone want to join me? So we will work together and learn some advanced knowledge.
the concept I am presenting is only from my mind, You can suggest some key mistakes and tell me which algorithms would work better and tell me which would be the best way to produce a Web App and deployment.