Career Profile
Product Engineer with specific focus on building robust backend architectures, highly resilient and scalable systems.
Skills & Proficiency
Programming Languages
Python
Golang
C
Web frameworks
Django
Flask
Databases
PostgreSQL
MySQL
Redis
Casssandra
CouchDB
ElasticSearch
Queues
AWS Kinesis (stream, firehose, analytics)
AWS SQS
Redis
Apache Kafka
Deployment & Infra
Amazon Web Services (AWS)
Docker
Elastic Container Service (ECS)
Google Cloud
AWS Serverless
CI and CD
Miscellaneous
Git
*nix systems
I.O.T
Clean, modular and testable code
Public Speaking/Tech talks
Hiring
Google search skills
Experiences
I joined here as one of the core backend engineer. I helped in abstracting and improving a large monolithic Artificial Intelligence platform into microservices architecture. This paved the way for MadStreetDen to build a full scale e-commerce suite of products called vue commerce.
After building the A.I platform, I started architecting products atop the platform. In this company, I have experienced the breaking down of large architecture to small microservices and scaling up of small 3 members engineering team to large 10 members one.
I helped Serendio in architecting and coding backend infrastructures to handle big data. I designed and developed real-time data pipelines to handle large stream of data and deployed them in private cloud. I also played a major role in hiring engineers and guiding junior developers to build the team.
Coming from electronics background, I joined here as a rookie developer and worked my way towards becoming a Data Engineer. In this company I got to know what is software engineering and how production systems work. My career switch from an embedded engineer to a software engineer happened here.
I designed and programmed firmware for industrial panel meters. Being an embedded programmer in the R&D team, I also developed many tools to automate the testing of tens and hundreds of meters which saved the time in production and man power.
Projects
- One of the core engineer who helped in abstracting a large, monolithic A.I platform into microservices architecture.
- I contributed in building this platform by architecting and maintaining multiple services like Database wrappers, Data ingestion servers and Serverless logging framework.
- Highlights: SOA, Microservices, Python, Django, Golang, AWS, Serverless, PostgreSQL, Docker
- Closely worked with Product design and Frontend teams to architect the APIs and Database schema for this product.
- Shop The Video is an A.I powered video marketing tool that helps the customer to monetize their video assets. It enables the purchase of the fashion items appearing on the video, directly form the video player.
- Designed and implemented the backend architecture from handling videos, catalog uploads to transcoding the videos and serving them through CDN.
- Highlights: Python3, Django, Serverless, AWS ECS, lambda, Athena, AWS Transcoder, Docker, Redis, PostgreSQL.
- Implemented a logging client with OOP paradigms in mind, which is actively being used across all the microservices developed in MadStreetDen.
- The object oriented approach helped in distributing the ownership of data integrity among all owners of the microservices.
- Implemented a serverless backend for this logging framework.
- Highlights: Python, OOP, AWS Athena, AWS Kinesis, AWS quicksight, AWS s3, Redis.
- Developed a REST API driven microservice around PostgreSQL database.
- This helped in rapid iterations of schema changes. Even though there were multiple microservices using the database, only this microservice had to be touched in case of schema change.
- Was able to derive more insights about slow queries and traffic patterns that affects the DataBase in API layer itself.
- API versioning helped in canary deployment for new schema roll outs.
- Highlights: Python, Django, Psycopg2, ORM, uWSGI.
- Built an I/O heavy, multi-tenant microservice that sits in ingestion part of the pipeline.
- This routes the data (products from the catalog) to different microservices and scales horizontally.
- Used multi threading to achieve the throughput of 10K messages per minute.
- Next version is being implemented using Golang, keeping concurrency in mind.
- Highlights: Python, multi-threading, concurrency.
- Developed a framework to ingest and parse product catalogs from various sources like HTTP, FTP, AWS SQS, S3.
- Implemented in a Object Oriented way which helped in writing custom parser for different e-commerce clients.
- Highlights: Python, multi-threading, multi-processing.
- Kulay is an high speed message passing system between various queues and services.
- It is a simple command line tool that runs as a daemon with a pretty heavy backend. One can route message packets from various services say like from Redis queue to Kafka, AWS SQS to redis etc.
- This project is in it's beta version in my github repository.
- Highlights: Golang, command line, Concurrency, Travis.
- I architected and developed a real-time data pipeline.
- This pipeline ingests high volume of twitter stream (5000+ tweets/min), extract information on the fly using NLP and expose the stored data via REST API. This pipeline was deployed in bare metal servers.
- Highlights: Python, Java, Kafka, Storm, Cassandra, Elasticsearch, Monit
- Developed an architecture to handle streaming data from Chinese websites.
- This service streams data from various Chinese websites like sina weibo, tencent, renren, etc. enriches it on the fly and stores them.
- Highlights: Python, Kafka, Cassandra, ElasticSearch, Solr.
- A Go clone of Redis which can be used as a library in your own code.
- This project is in it's (discontinued)beta version in my github repository.
- Highlights: Golang, Concurrency, Semaphore, Mutex, Redis
- Built a multi pin socket box (spike buster), where devices connected in each sockets can be controlled and monitored from Internet.
- A responsive webapp was built with an interface to toggle connected devices. It also shows the energy readings of each devices connected in socket.
- Highlights: IoT, Python, Flask, Arduino, Microcontrollers, SQLite.
- Developed a 6 compartment pill box that is connected to Internet. Once the details of pills in each compartment is setup in the web interface, it reminds the user at the given time to take the particular pill. It also sends an email to place an order when any pills gets over.
- Highlights: IoT, Python, Flask, Arduino, Microcontrollers, SQLite.