Pivotal Engineering Journal
Technical articles from Pivotal engineers.
Posts By Swati Soni
Mar 24, 2018
Scaling Machine Learning to Recommend Driving Routes
The Pivotal Data Science Labs helped a multinational customer build a scalable, real-time predictions and recommendations application to increase revenue. We built an end-to-end machine learning workflow which addresses online deployments and offline training using open source projects and Pivotal products.
Spark & Hadoop
real-time prediction & recommendation
feature engineering & modeling pipeline
driver revenue prediction
Scale Machine Learning as a service
Feb 1, 2017
Agile Development for Highly Scalable Data Processing Pipelines
Legacy data processing pipelines are slow, inaccurate, hard to debug, and can cause thousands of dollars in revenue. Conforming to agile methodology and a detailed seven-step approach can ensure an efficient, reliable and high-quality data pipeline on distributed data processing framework like Spark. Learn how following TDD, careful creation of data structures, and parallel execution results in: code competency and completeness, and a linearly or constantly scalable robust big data processing pipeline.