Working on various Python projects has taught me the importance of consistent logging, especially when dealing with distributed computing frameworks like Spark. Logging is not just about keeping track of errors or information; it’s about having a detailed and systematic record of the operations to understand the flow of your program and quickly diagnose issues.… Continue reading My Python logging setup
Tag: python
Readability scoring of the United Nations Corpus
Imagine you could estimate how hard would be to read a document, before reading it. Imagine you could do it for entire batches of documents you need to process. Imagine you could have a recommender system that would help you prioritize unread documents according to their difficulty. A bit of experimentation with the public United… Continue reading Readability scoring of the United Nations Corpus
Recommender system for finding subject matter experts using the Enron email corpus
This is a little project to create a recommender system to find mentors inside an organization, using Natural Language Processing. It started as an excuse to build a data visualization I had in mind: an interactive word cloud that did something. When I started, I didn’t know anything about Topic Modeling, Topic Extraction, or Natural… Continue reading Recommender system for finding subject matter experts using the Enron email corpus
Linear optimization with or-tools: containerizing a gunicorn web application
Previously, we left our app working with our local python+gunicorn+nginx installation. In order to get there we had to do quite a bit of configuration and if we wanted to deploy this in a server or send it to a friend, we would have to go through a very error-prone process subject to version changes… Continue reading Linear optimization with or-tools: containerizing a gunicorn web application
Linear Optimization with or-tools — building a web front-end with falcon and gunicorn
In a previous post, I put together a script for solving a linear optimisation problem using Google’s OR-tools. This python script is callable from the command line and you kinda need to know what you are doing and how to organize the parameters. So, in order to address this difficulty, I wanted to build a… Continue reading Linear Optimization with or-tools — building a web front-end with falcon and gunicorn