A real-life example using Scrapy and Github Actions

Image for post
Image for post
Photo by Marten Newhall on Unsplash

Job hunting is a time-consuming task. A lot of different sites for job searches exist, but there is not a “one size fits all”. Job openings are available in job aggregators, LinkedIn, career pages of individual companies, even as tweets or in Git repos. Following all the changes is definitely challenging.

But what if you could build your personal job hunting tool? This post is exactly about this. With the help of scrapping tools we’ll build a small proof-of-concept that helps you keep track of jobs posted to company web sites. The data will be extracted in JSON format. This way you can build your own personalized newsletter or career page. …

Or how to automate your machine learning training pipeline using Git and a CI server.

Image for post
Image for post
Photo by tian kuan on Unsplash

A machine learning pipeline is more than some tools glued together. It is a process that can make the lives of data scientists and developers easier by removing any mundane task or error-prone step. The automation of the training process requires the collaboration of many different team members The term MLOps has been used to describe the combination of software engineering, machine learning and system operations in order to bring models to production. Platforms for managing machine learning pipeline already exist. In the next paragraphs, a solution using Git and a CI/CD as the core of the pipeline is presented.

The reasons why this article is being written…


Ioannis Foukarakis

Software engineer, interested in machine learning projects and process, back-end development and coding in general. Geek since before it was cool to be one.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store