featuretools

I worked on building a python library for automated feature engineering called Featuretools (https://github.com/featuretools/featuretools/). I had been working on it for 2 years, but in 2017 we separated it from the rest of the codebase and made it open source. Even though feature engineering is crucial for building machine learning pipelines, there are few formal methods for performing feature engineering. We see Featuretools filling a missing component in the software engineering stack for data science. It has already been put to the test with our customers at my company, but we have also begun to release demos so that others can pick it up https://www.featuretools.com/demos.

This is absolutely correct! After spending a lot of time trying to understand why building machine learning models is so difficult, I came to the same conclusion that "feature engineering" is the key to building high performing models. While feature engineering's importance is generally recognized [0], it's unfortunate that there aren't more tools and formal methods for applying it. Personally, I am a developer of an open source python library called Featuretools (https://github.com/featuretools/featuretools/) that is trying to change this for tabular and multi-table datasets. We are working hard to make automated feature engineering available to everyone and have a list of demos for people to try here: https://www.featuretools.com/demos. It's also worth noting that deep learning is changing the need for feature engineering. However, it p

As someone who works with a lot of people new to machine learning, I appreciate guides like this. I especially like the early slides that help frame AI vs ML vs DL so that people can have a realistic understanding of what these technologies are for. For my part, one of the biggest realization I had after many years of applying machine learning was that I got too caught up in the machine learning algorithms themselves. I was often way too eager to guess and check across different algorithms and parameters in search of higher accuracy. Fortunately, there are new automated tools today that can do that automatically. However, the key piece of advice I'd give someone new to machine learning is not to get caught up in the different machine learning techniques (SVM vs random forrest vs neural network, etc). Instead (1) spend more time on transla

the latest release is for python 2.7 only. we have a pull request [0] up right now for python 3 that passes all the automated testing. [0] https://github.com/Featuretools/featuretools/pull/16

It somewhat depends on if you are looking to build AI to address business problems or if you are more interested in the type of AI work you see companies like Google discussing. I can speak to what "AI" means for most businesses outside Top Tech which more frequently work with tabular, relational, or log data rather than image and text. For these companies, this is what you need to learn how to do 1. Define a prediction problem and extract labels 2. Organize and clean the data for prediction 3. Perform feature engineering by applying domain expertise 4. Apply an off-the-shelf open source machine learning algorithm like a random forest Assuming you have access to data and programming skills to clean your data, defining prediction problems and performing feature engineering are the most important skills you have to pick up.

I think this highlights the disconnect between AI researchers and industry. This research focuses on image data and then makes the claim that they want to use it to enable millions of businesses to solve problems with machine learning. However, image data isn't what most businesses are using for machine learning. They use relational datasets. If you look at the recent "State of Data Science" by Kaggle, relational data is ~3x more common than image data in every industry besides academia and the military [0]. While Google wants to 1000x the number of organizations using AI, they aren't focusing on the problems companies actually have. Basically, academics love building AI for images, but what companies really need are better ways to build AI systems on tabular and relational data. Images will be a piece, but shouldn't be the focus. Disclai