BEbelter·Sep 20, 2023Submission
Featuretools – A Python Library for Automated Feature Engineering
1 points · Open discussionalteryx
featuretools
Python2017–2023
alteryx / featuretools
featuretools
An open source python library for automated feature engineering
Learning & ReferenceScience & EngineeringAI & Machine LearningAutomation & WorkflowsLibraries & SDKs
7Distinct people
38Mentions
1Explicit recommenders
7.7KGitHub stars
What to know before exploring
What it is
An open source python library for automated feature engineering
GitHub
Where it is used
No documented product use has been verified yet.
What people are saying
To get started, I'd pick the most important business problem you have and then solve it using the simplest machine learning approach. You mentioned using Pytorch. Instead, I…
kmax12 used a recommendation phrase. 2 indexed comments match. This is a phrase match, not a verdict.
No classified warning is attached to this page. Read the discussions before treating silence as approval.
Also in these discussions
- alteryx/compose · linked in 2 of the same discussions
Mentioned in the same Hacker News threads. That is not proof they are substitutes.
This week on Hacker News
Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 3:56:07 PM UTC.
No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.
Mentions over time
HN mentions by year38 dated mentions
Observed mentions: 2017-11-06 14:51:21 UTC to 2023-09-20 21:40:24 UTC.
Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.
View mention counts and date ranges
| Period | Start (UTC) | End (exclusive, UTC) | Mentions | Submissions | Comments |
|---|---|---|---|---|---|
| 2017 | 2017-01-01 00:00:00 UTC | 2018-01-01 00:00:00 UTC | 7 | 1 | 6 |
| 2018 | 2018-01-01 00:00:00 UTC | 2019-01-01 00:00:00 UTC | 19 | 0 | 19 |
| 2019 | 2019-01-01 00:00:00 UTC | 2020-01-01 00:00:00 UTC | 7 | 0 | 7 |
| 2020 | 2020-01-01 00:00:00 UTC | 2021-01-01 00:00:00 UTC | 4 | 0 | 4 |
| 2021 | 2021-01-01 00:00:00 UTC | 2022-01-01 00:00:00 UTC | 0 | 0 | 0 |
| 2022 | 2022-01-01 00:00:00 UTC | 2023-01-01 00:00:00 UTC | 0 | 0 | 0 |
| 2023 | 2023-01-01 00:00:00 UTC | 2024-01-01 00:00:00 UTC | 1 | 1 | 0 |
What Hacker News says
38 sourcesAs another option, Compose is a ML tool for labeling data. It structures the labeling process and integrates easily with Featuretools which automates feature engineering. Definitely worth checking out!
[1] https://github.com/FeatureLabs/compose
[2] https://github.com/FeatureLabs/featuretools
In Snorkel AI: Putting Data First in ML DevelopmentI know I'm not the only one, but it's hard to imagine doing my job the last several year without Pandas. Even though Pandas has been used in production by many people as basically a 1.0.0 release for a long time, this an amazing milestone and I think everyone in my office smiled when they saw the release news.
I think it's worth it to acknowledge the great stewardship of the community by all the Pandas developers (and the rest of people in the PyData ecosystem). It has been an inspiration for me as I create and contribute to open source libraries for data science [0][1].
[0] https://github.com/FeatureLabs/featuretools/
[1] https://github.com/FeatureLabs/compose/
In Pandas 1.0Part of the problem is that if you gave 20 developers/data scientists/ml engineers the same the set of data and asked them the do data prep and feature engineering, you'd probably have them come back with 20 different approaches.
To avoid pipeline jungles, teams need to agree to certain API's that their data processing code will follow e.g scikit-learn helped many people standardized around fit/predict/transform for their machine learning algorithms. In the future, I expect we'll see this expand to other parts of the process, such as feature engineering.
Towards that goal, I work on an open source library trying to do this for feature engineering called Featuretools. You can check it out here: https://github.com/FeatureLabs/featuretools/
In Technical Debt in Machine Learning Systems (2015) [pdf]Feature Labs, Inc. | Software Engineer | Boston, MA | Full-time | On-site | https://www.featurelabs.com
At Feature Labs, we’re building automation technology to make machine learning easier to use. We work on many unique and important machine learning challenges with a focus on making sure machine learning isn’t just a theoretical endeavor but has a real-world impact.
Currently, our stack is mostly Python on the backend (pandas, django, sklearn), React for the front end, and AWS and docker for deployment. Above all, we prioritize the ability to choose the right approach to solve a problem.
We love open source and hope you do too: https://www.featurelabs.com/open
Here is one of our most popular libraries: https://github.com/featuretools/featuretools
Apply at https://www.featurelabs.com/careers/
In Ask HN: Who is hiring? (January 2020)Feature Labs, Inc. | Software Engineer | Boston, MA | Full-time | On-site | https://www.featurelabs.com
At Feature Labs, we’re building automation technology to make machine learning easier to use. We work on many unique and important machine learning challenges with a focus on making sure machine learning isn’t just a theoretical endeavor but has a real-world impact.
Currently, our stack is mostly Python on the backend (pandas, django, sklearn), React for the front end, and AWS and docker for deployment. Above all, we prioritize the ability to choose the right approach to solve a problem.
We love open source and hope you do too: https://www.featurelabs.com/open
Here is one of our most popular libraries: https://github.com/featuretools/featuretools
Apply at https://www.featurelabs.com/careers/
In Ask HN: Who is hiring? (October 2019)Feature Labs, Inc. | Software Engineer | Boston, MA | Full-time | On-site | https://www.featurelabs.com
At Feature Labs, we’re building automation technology to make machine learning easier to use. We work on many unique and important machine learning challenges with a focus on making sure machine learning isn’t just a theoretical endeavor but has a real-world impact.
Currently, our stack is mostly Python on the backend (pandas, django, sklearn), React for the front end, and AWS and docker for deployment. Above all, we prioritize the ability to choose the right approach to solve a problem.
We love open source and hope you do too: https://www.featurelabs.com/open
Here is one of our most popular libraries: https://github.com/featuretools/featuretools
Apply at https://www.featurelabs.com/careers/
In Ask HN: Who is hiring? (September 2019)Feature Labs, Inc. | Software Engineer | Boston, MA | Full-time | On-site | https://www.featurelabs.com
At Feature Labs, we’re building automation technology to make machine learning easier to use. We work on many unique and important machine learning challenges with a focus on making sure machine learning isn’t just a theoretical endeavor but has a real-world impact.
Currently, our stack is mostly Python on the backend (pandas, django, sklearn), React for the front end, and AWS and docker for deployment. Above all, we prioritize the ability to choose the right approach to solve a problem.
We love open source and hope you do too: https://www.featurelabs.com/open
Here is one of our most popular libraries: https://github.com/featuretools/featuretools
Apply at https://www.featurelabs.com/careers/
In Ask HN: Who is hiring? (August 2019)I think it’s a bit of an overstatement to call this an end-to-end solution.
What they are starting with here is a single table of data with all the features already defined and an existing binary label column. Typically when this type of data is collected in the field it is much more fine grained (i.e many observations collected over time) and unlabeled (e.g how do we define a true example? How many false examples do we select?).
The competition description even goes as far to say “We have chosen a dataset that you can get started with easily”.
So, yes, this is a cool demonstration of Google's product, but the success in the competition might not extend to the problems real business face when trying to apply ML to a problem like this.
That being said, I do think AutoML can help with these problems as it is extended to handle data that isn
Read the rest
’t in a single table already.
For example, I’m a developer of a open source library called Featuretools (https://github.com/Featuretools/featuretools) that tries to automate feature engineering for temporal and relational datasets. Basically, it helps data scientists prepare real world data into the form this competition starts with.
I see this as a very timely question. As ML has proliferated, so has the number of ways to construct machine learning pipelines. This means that from one project to another the tools/libraries change, the codebases start looking very different, and each project gets its own unique deployment and evaluation process.
In my experience, this causes several problems that hinder the expansion of ML within organizations
1. When a ML project inevitably changes hands, there is a strong desire from the new people working on it to tear things down and start over
2. It's hard to leverage and build upon success on previous projects e.g "my teammate built a model that worked really well for predict X, but I can use any of the that code to predict y"
3. Data science managers face challenges tracking progress and managing multiple data science projects
Read the rest
simultaneously.
4. People and teams new to machine learning have a hard time charting single a path to success.
While coming up with a single way to build machine learning pipelines may never be possible, consolidation in the approaches would go a long way.
We've already seen that happen in the modeling algorithms part of the pipeline with libraries like scikit-learn. It doesn't matter what machine learning algorithm you use, it the code will be fit/transform/predict.
Personally, I've noticed this problem of multiple approaches and ad-hoc solutions to be most prevalent in the feature engineering step of the process. That is one of the reasons I work on an open source library called Featuretools (http://github.com/featuretools/featuretools/) that aims to use automation to create more reusable abstrations for feature engineering.
If you look at our demos (https://www.featuretools.com/demos), we cover over a dozen different use cases, but the code you use to construct features stays the same. This means it is easier to reuse previous work and reproduce results in development and production environments.
Ultimately, building successful machine learning pipelines is not about having an approach for the problem you are working on today, but something that generalizes across the all the problems you and your team will work on in the future.
Deep feature extraction is important for not only image analysis but also in other areas where specialized tools might be useful such as listed below:
o https://github.com/Featuretools/featuretools - Automated feature engineering with main focus on relational structures and deep feature synthesis
o https://github.com/blue-yonder/tsfresh - Automatic extraction of relevant features from time series
o https://github.com/machinalis/featureforge - creating and testing machine learning features, with a scikit-learn compatible API
o https://github.com/asavinov/lambdo - Feature engineering and machine learning: together at last! The workflow engine allows for integrating feature training and data wrangling tasks with conventional ML
o https://github.com/xiaoganghan/awesome-feature-engineering - other resource related to feature engineering (video
Read the rest
, audio, text)
To add to the three points in the article:
Data quality
https://en.wikipedia.org/wiki/Data_quality
Imputation
https://en.wikipedia.org/wiki/Imputation_(statistics)
Feature selection
https://en.wikipedia.org/wiki/Feature_selection
datacleaner can drop NaNs, do imputation with "the mode (for categorical variables) or median (for continuous variables) on a column-by-column basis", and encode "non-numerical variables (e.g., categorical variables with strings) with numerical equivalents" with Pandas DataFrames and scikit-learn.
https://github.com/rhiever/datacleaner
sklearn-pandas "[maps] DataFrame columns to transformations, which are later recombined into features", and provides "A couple of special transformers that work well with pandas inputs: CategoricalImputer and FunctionTransformer"
https://github.com/scikit-learn-contrib/sklearn-pand
Read the rest
as
Featuretools
https://github.com/Featuretools/featuretools
> Featuretools is a python library for automated feature engineering. [using DFS: Deep Feature Synthesis]
auto-sklearn does feature selection (with e.g. PCA) in a "preprocessing" step; as well as "One-Hot encoding of categorical features, imputation of missing values and the normalization of features or samples"
https://automl.github.io/auto-sklearn/master/manual.html#tur...
auto_ml uses "Deep Learning [with Keras and TensorFlow] to learn features for us, and Gradient Boosting [with XGBoost] to turn those features into accurate predictions"
https://auto-ml.readthedocs.io/en/latest/deep_learning.html#...
To get started, I'd pick the most important business problem you have and then solve it using the simplest machine learning approach.
You mentioned using Pytorch. Instead, I recommend a classical machine learning using a library like scikit-learn (https://scikit-learn.org/). Use a random forest classifer and you'll get pretty good results out of the box.
If your data is in a postgres database across multiple tables, you will likely have to perform feature engineering in order to get it machine learning ready. For that, I recommend a library for automated feature engineering called Featuretools (http://github.com/featuretools/featuretools/). Here's a good article to get started with it (https://towardsdatascience.com/automated-feature-engineering...)
Finally, you will need to define a prediction problem and extract labeled training example
Read the rest
s. I see people in this thread have suggested ideas of problems to work on. The key here is make sure that you pick a problem that you can both predict and take an action based off the prediction. For example, you could predict that there will be an influx of shipments to fulfill tomorrow, but that might not be enough to time to hire more people to help you fulfill them.
If you're curious what the process looks like end-to-end check out this blog series on a generalized framework for solving machine learning problems that was applied to customer churn prediction: https://blog.featurelabs.com/how-to-create-value-with-machin...
Full disclosure: I work for Feature Labs and develop Featuretools.
I definitely see the need for packages like this. So much of a data scientist's time is spent on feature engineering, but there are relatively few tools out there that are trying to improve that step in the process compared to tools for the modeling step.
I see this tool as a something that can help with the deployment piece of the feature engineering. As things stand, it's "easy" to package and deploy modeling code, but much harder to package up your feature engineering workflow. In part because there is no agreed upon standard for developing feature engineering pipelines.
I'd be curious how this could be combined with a library like Featuretools (http://github.com/featuretools/featuretools/) which helps automate the discovery of features, but currently has less functionality related to deployment.
(full disclosure: I work on Featuretoo
Read the rest
ls)
Feature Labs, Inc. | Software Engineer | Boston, MA | Full-time | On-site or Remote | http://www.featurelabs.com
Feature Labs is changing the way companies create new machine learning products and services. We make a web app and developer API to automate time-intensive and error-prone parts of the data science process such as feature engineering. Our customers love our products because they make machine learning easier to use.
Currently, our stack is mostly Python on the backend (pandas, django, sklearn), React for the front end, and AWS and docker for deployment. Above all, we prioritize the ability to choose the right approach to solve a problem.
If you're interested in getting hands on with our product, check out our open source library, Featuretools https://github.com/featuretools/featuretools/.
Apply at https://www.featurelabs.com/ca
Read the rest
Showing the 15 most recent of 38 sources. Enable JavaScript to search and browse every indexed mention.