This is Sumit Agarwal

A Social Animal

A B.Tech Graduate from the department of Computer Science and Engineering, Indian Institute of Technology,Kharagpur, I keep a knack in programming especially in areas of machine learning, deep learning and natural language processing. A foodie at heart, I also love to act, elocute, sing, dance, click pictures and an admirer of Hindi prose and poetry.

खुद ही को कर बुलंद इतना हर तक़दीर से पहले,
कि ख़ुदा खुद बन्दे से पूछे बता तेरी रज़ा क्या है |

Make yourself so capable enough so that before writing your destiny, even the almighty asks you what do you want as your fortune? [English Transalation]

The Kgp Intro

My name is Sumit Agarwal (The customary answer to what is my name). I am a graduate student from the department of Computer Science and Engineering enrolled in its B.Tech course (The kind of student I am undergrad/grad followed by the department name and the degree I am pursuing). I am from Calcutta (The place where I come from and it should be only the city name). My hobbies are acting, elocution and choreography (These were supposed to be from those things that were part of the inter hall/hostel competition that you had interest in to let others know that you have either participated in it or want to participate).

Multi Device Voice Assistant Experience

Working towards enabling consistent voice assistant experience for users across devices like speaker, mobile, television, refrigerator. Through intelligent device selection help the user smoothly transition among devices. For eg, when the user asks the speaker to “What’s the weather”, the speaker can not only speak out the weather but also show the detailed view on the television.

Bixby Capsule Developer

Developing voice enabled apps called capsule for Bixby, capable of teaching capabilities to perform a variety of tasks. (https://bixbydevelopers.com). Currently working on clock, messages and settings capsule for mobile and speaker devices and achieved an accuracy of over 95% consistently on production usage.

Dynamic conversation drivers

To achieve the goal of generating variant utterances under the same semantic frames, the input utterance is converted into its delexicalised form with its slot values replaced by labels, and then generating its delexicalised variances with a seq2seq model. Finally, surface realization is carried out to convert the delexicalised form into the raw utterance. Used BERT for next step classification to remove false positives, by matching the intents of the output with the input and achieved multiplication rate of 2, i.e., generating 2 utterances for every utterance in the training set.

Data augmentation for dialog systems

Implemented a system to suggest dialogs to drive the conversation forward with the user. For example, suggesting the user to “Book an Uber” when he asks his “Direction to work”. Implemented the system as a multi label classification problem which used the user’s utterance as input to the sequence to sequence model where every step of the decoder was used to generate labels/next intent of the user. These intents were further used to suggest dialogs. Training data being less, achieved an improvement in F1-score of 5% on using pre-trained BERT as encoder and fine-tuning it with the training data.

Low-corpus classification for voice assistants

Used a three-step classification pipeline, domain classification, then intent classification followed by slot labelling using BERT. Fine tuned the model with low corpus data from Bixby across 10 different domains and achieved an accuracy of 95% on domain and intent classification, and 85% on slot tagging..

Identifying data clashes in Bixby

To identify the classification clashes across Bixby capsules, extracted similar utterances trained across capsules using tf-idf and cosine similarity. Listed the most important words in each capsule through tf-idf scores to help developers get a deeper insight in their capsule’s training data, balancing and modifying the data wherever necessary.

On-device light-weight NLU for voice assistants

Wrote a fast-text based text classification for dialog systems with identification of domain, intent and slot values which was light-weight meant to be run on devices like television, refrigerator, washing machine. The model had an accuracy over 90% and performed better than the rule-based technique used before.

Samsung R&D Institute Kharagpur, India

Software Developer Intern 2016

Worked on sms mining and extracting information from transaction messages using basic classification and NLP techniques. Worked with the server team to integrate the UPI(United Payments Interface) service, a real-time system made by the Government of India, with the Samsung Pay wallet.

National Digital Library Kharagpur, India

Software Developer Intern 2015

Worked on building a search engine with multi-lingual support for the National Digital Library using Apache Solr as the backbone. Configured a multilingual frontend having phonetic and keyboard input using Google API’s. Using stemmers enabled searching and retrieving documents across ten Indian languages. Worked on language detection that needed to be checked before applying the respective stemmer. The project was sponsored by the Ministry of Human Resources Department, Govt. of India.

Social Opinion Dynamics

Prof. Niloy Ganguly CSE,IIT KGP

Predicting future opinions of users in a social network by considering the influence of its friends. Used network-guided recurrent neural networks to model the time and frequency of the opinions of the users as opinion dynamics has an inherent recursive structure depending on past opinions. Achieved an improvement of 10% in the MSE of the predicted opinions over previous state-of-the-art models as the non-linearities of RNNs captured the opinion exchange process concretely. Paper titled Learning Non Linear Opinion Dynamics in Social Networks accepted in International Conference on Data Mining(ICDM'17).

Commbox

Prof. Niloy Ganguly CSE,IIT KGP

Commbox is a real time commentory box which uses machine learning techniques and real time sensor data (accelerometer, gyroscope) from wearable devices. It analyses the sound of the ball hitting the bat by distinguishing it from any other sound by fixing a threshold of the sound intensity to start recording the data from the wearable devices which then goes into a classifier to predict the shot played. The project recieved the best demo paper award at COMSNETS'17. Paper titled Learning Non Linear Opinion Dynamics in Social Networks accepted in International Conference on Data Mining(ICDM'17).

Vehicular Environment Detection

Prof. Pabitra Mitra CSE,IIT KGP

Developing on-board automotive driver assistance systems for detecting environments to help the drivers and also applications in autonomous driving . We focus on assisting the driver or the autonomous vehicle to either turn left, right or go straight by the images that it captures from its front camera. Images collected from the popular video game Road Rash for Up, Right, Left through logging key press events. Applied AlexNet architecture based on Convolutional Neural Networks for classifying the images.

Scientific Article Summarization

Prof. Pawan Goyal CSE,IIT KGP

Worked on a text summarisation task for automatic abstract generation for scientific documents. Converted the paper to paragraph embeddings using para2vec and trained a bidirectional sequence to sequence LSTM network with attention to capture the long-term dependencies in scientific papers. Achieved ROUGE-1 scores of 0.5. Although, results weren’t stunning, could show that LSTMs have the potential to work for long documents but require more computational power.

Citation Recommendation System

Prof. Pawan Goyal, Prof. Animesh Mukherjee CSE,IIT KGP

An information retrieval system to fetch papers which can be cited given a citation context as a query. Compared the search results across various combinations of weights assigned to different parts of the paper (title, abstract, citation context) and used Tf-Idf and BM-25 to retrieve the papers. Used query expansion by expanding the query with similar words using word2vec. BM-25 with title given the highest priority performed the best among all with a MRR (Mean Reciprocal Rate) of 0.2 over a test-set of 1000 documents.

Bus Tracker Application

Prof. Pabitra Mitra CSE,IIT KGP

Application to help students track buses plying in the campus and know the times of buses at various bus-stops through the bus’ GPS information. Duplicated the bus with a mobile application as a substitute for GPS device and made another device to track the location of these mobiles (or buses) to simulate the project in a classroom environment.