Java Spring Boot MySQL Apache Kafka Microservices Docker Kubernetes Kibana Graphana CI/CD Jenkins
Python 3 Keras BERT CNN LSTM Jupyter notebook
Angular 4 Flask HTML CSS JavaScript Python 3 GCP Plot.ly
Details how we were able to scale our existing Ads.Txt Crawler using Apache Kafka technology to support the increasing growth at PubMatic.
30 Mar 2023 • 5 min readThe Ads.Txt Crawler 2.0 effectively scans ads.txt files across 7 million domains. The project involved the creation, management, and maintenance of two microservices with Kafka producers and consumers. The project also entailed testing various text compression algorithms, eventually choosing one that compressed message payload size by 95%.
HTML CSS Javascript Bootstrap 4 Python 3 Django REST API
The Ads.Txt Crawler 2.0 effectively scans ads.txt files across 7 million domains. The project involved the creation, management, and maintenance of two microservices with Kafka producers and consumers. The project also entailed testing various text compression algorithms, eventually choosing one that compressed message payload size by 95%.
HTML CSS Javascript Bootstrap 4 Python 3 Django REST API