Atelier API
Repository🌟 Project Overview
Situation: Refactoring a Node.js RESTful API deployed on AWS EC2, serving retail product data from a PostgreSQL database.
Task: Optimize and scale the API to handle production level traffic with low latency and <1% error rate. This backend was crucial to have ready for the next project, which was the frontend serving this data. It was a challenge without using a Kubernetes cluster or database sharding.
Action: Dockerized the API for horizontal scaling with the NGINX proxy service. Used the dynamic least connection algorithm for load balancing to distribute traffic. Optimized the Node.js app with caching, refactoring database queries, and other minor code improvements. Identified and resolved system bottlenecks with load testing using K6 and Loader.io.
Result: Achieved traffic from <10 RPS to reliable 700 RPS with 70 ms average latency and <1% error rate. Avoided breaking changes with testing and continuous integration, decreasing the stress testing cycle length to allow quick iteration of the system..
📈 Performance Results
Target Performance
- Throughput: 100 RPS
- Latency: 2000 ms
- Error rate: <1% rate
Actual Performance
- Throughput: ~700 RPS (Max 5000 RPS)
- Latency: ~70 ms
- Error rate: <1% rate
Engineering Journal
To keep track of everything I learned, along with the challenges faced during this project, I maintained an Engineering Journal available as a PDF, Markdown, or Google Docs
API Documentation
The endpoints for this project are documented in this markdown file. This project was dedicated to rebuilding the Products module of the Atelier API.
📦 Technologies:
- NGINX
- Docker
- AWS EC2
- Postgres
- Node.js / Express
- Jest
- GitHub Actions
- Loader.io
- K6
- Sequelize
- Axios
- ESLint
- Husky
- Prettier
Extra Resources:
- API Documentation: Atelier API
- Engineering Journal: PDF Markdown Google Doc
- Ticketing System: Trello Board:
- Source Code: GitHub Repo
Also find me here: