Keynote | Technical
Thursday 16th | 15:55 - 16:25 | Theatre 18
Keywords defining the session:
- Data enrichment
- Near real-time
In order to fulfill increasing needs of business to deliver rich and near real time data for various analysis of Web users behaviour, the main and complex Apache Spark process was replaced in most parts by more modular and reusable solution built on top of Apache NiFi. Due to this transformation a significant boost and stability of entire process was achieved along with decrease of resources consumption. Also a few approaches were made to ingest the data to Hive in small batches.