Description
This marketing analytics tool is focused on SEO A/B testing. It scans a website and lists out pages that have the potential to increase their click-through rate (CTR) and ranking on Google's SERP. Then, the app enables the website owner to test how changes to the meta description and title tag, and body content can affect the ranking of the page.
Role
The success of that cloud SEO tool depended on getting daily, performant crawl statistics from a considerable number of websites, while the app was written in Ruby and, at the time, wasn’t quite suited to handle large scale distributed crawling. The goal of our Puppeeteer/Node.js crawler microservice was to process websites and notify the Ruby app once the job is done. Ruby then processed the crawl and made all the required adjustments in the app. Our developers also used RabbitMQ for caching and saving temporary results.
