The aim of this short blog article is to show you how to migrate data from a Data source that you can connect to via Spark (HDFS for instance) to an ElasticSearch index by leveraging the Elastic-hadoop driver. We will start by a short presentation of ElasticSearch and Spark frameworks and the possible use cases of these two famous Data Engineering tools and then we’ll move forward to the demo.
In this article, I provide feedback about my experience with AWS Data Analytics Specialty Exam.
Take this exam if you want to boost your Data Engineer / Analyst career within the world of AWS or any other cloud provider. The high-level concepts learned while preparing for the ceritifcation can be easily applied to other providers as well.
Take your Spark Big Data workloads to the next level by leveraging the power and flexibility of Kubernetes.
In this article, we are going to learn why it may be relevant to use Spark on Kubernetes and how to do it.
For the sake of our demo we will be using a Minikube Cluster to run a basic Spark-Pi job.
Apache Spark is a framework that can quickly perform processing tasks on very large data sets, and Kubernetes is a portable, extensible, open-source platform for managing and orchestrating the execution of containerized workloads and services across a cluster of multiple…
In this article, I provide feedback about my experience with AWS Machine Learning Specialty Exam (which I cleared with 95%).
Take this exam if you want to boost your Data Science / Machine Learning Engineering career within the world of AWS or any other cloud provider. The high-level concepts learned while preparing for the ceritifcation can be easily applied to other cloud providers as well.
Since it’s first 1.x release, Spark became the de facto Big Data unified processing Engine. 9 of 10 companies chose Spark for their Data processing thanks to its speed, ease of use, modularity and extensibility. The aim of this series of articles is to:
We assume that you are already familiar with Spark Structured High Level…
La crise du Coronavirus a été pour nous tous une expérience inédite qui va laisser des traces à jamais. Pendant deux mois, nous avons renoncé à ce qu’il y a de plus fondamental dans notre mode de vie : Notre liberté de circulation. Nous y avons renoncé et nous nous sommes confinés chez nous, malgré nous. Plus de sorties entre amis. Plus de réunions familiales. Plus de balades. Plus de soirées. Plus de cafés ou de restos. Plus de vacances.
Heureusement pour nous, le virus disparait progressivement. Ses mutations le rendent de moins en moins dangereux. Le nombre de nouveaux…
I am currently Taking Deeplearning.ai Deep Learning Specialization which I highly recommend by the way for people who are looking for a great place to start deep learning. The Specialization is offered on Coursera and I have just finished their third Course : Structuring Machine Learning Projects.
The course explains how to be systematic when it come to thinking about Deep Learning projects and gives you an array of tools that’ll help you to make the right decisions and move forward with your machine learning project.
Most of the ideas of this course are not included in university deep learning…
In this tutorial, you will learn how to build your own Meme Detector from scratch like this one. By the end of the tutorial, you will be able to: