Advisor : Dawn Song

Objective : Neural symbolic hybrids for few shot recognition

• Using program induction to sample programs for few shot image classification.

• Training procedure involves Supervised pre-training with teacher-forcing followed by reinforcement learning using Hindsight Experience Replay.

• We use memory augmented networks with attention to allow multiple chains of execution.

Objective : Meta learning in SQL query synthesis

• Divided the Spider dataset into 13 meta categories.

• We use transformers to generate embeddings for natural language question tokens.

• A meta-training phase for decoder to learn predicting the structure of SQL query

• A domain specific training phase for token prediction using a separate multi head attention module.

Advisor : Prof. Moritz Helmstaedter

Objective : Myelin segmentation in 3D mSEM and connectomic analysis

• We used 3D Unet trained on multi Scanning Electron Microscope raw data to generate 3D segmentation masks

• Dynamically oversampling (with linearly decaying probability) myelinated voxel cubes to provide non-zero gradients countering highly skewed data (0.01% positively labeled voxels)

• Reached over 90% precision-recall. Using the detection to analyze thalamocortical neurons with myelinated axons at the beginning of innervation.

Advisor : Prof. Tat Seng Chua

Objective: Monocular 3D object instance recognition and Pose Estimation

• Proposed (alongside a post graduate student) a novel end-to-end architecture consisting of two modules for robust pose prediction and 3D instance recognition via extracting Marr’s 2.5 D sketches from images.

• The learned embedding explicitly disentangles a shape vector and a pose vector, which alleviates both pose bias for 3D shape retrieval and categorical bias for pose estimation

• One sub module learns to reconstruct 3D model, from the 2.5D sketches, in its canonical viewpoint via multi-task learning DNNs. Another NN sub module uses Faster R-CNN style anchor boxes to predict the 6 DoF poses in continuous domain.

• The method achieves state of the art 10.3 median error for pose estimation and 0.592 top-1-accuracy for category agnostic 3D object retrieval on the Pascal3D+dataset.

Objective: Industrial grade deployment of ML backend and android application for NYO

• Lead a team of 16 people at NYO.

• ML systems:

  • Collaborative Filtering for Recommendation engine
  • Automated response collection from handwritten characters markings on response sheets for grading
  • NLU chatbot using RASA pipeline with NER, Relationship extraction and quantity association for extracting required information from user.

• Android app: REST APIs, SSE notifications, app-caching, Continuous integration with Jenkins, data and property binding, Reactive Java for observables