Dynamic data mining of advertising data from unknown sources

Purpose and goal

The goal of the project has been to automatically extract advertising data from previously unknown online sources using advanced crawling, natural language processing and machine learning. The resulting robot is presently capable of collecting data from a large number of this type of sources after a minor amount of training from our support staff.

Expected results and effects

Qwaya has seen the need from customers to automatically analyse and generate reports on spend and results of their online advertising. The large number of advertising platforms that do not provide a reporting API is a challenge in creating such a solution. The project has successfully delivered technology to extract advertising data from new online sources without an API. This has resulted in improvements to our product, Funnel, where we can now promise customers that we can analyze all their advertising spend and give them a complete picture of their marketing budget and results.

Planned approach and implementation

The project has covered topics such as crawling of dynamic, javascript- and ajax-based environments, classification of advertising data and interpretation of table- and graph-data. The results has been continually integrated in to the product Funnel, and evaluated using real world test data. With a strong focus on iterative improvements and user testing we have achieved a result that is well integrated in the product, while also providing a lot of insights into further possible developments of the technology.