DSdsr12·Sep 22, 2020Submission
Pandas just dropped Python 3.6 support
2 points · Open discussionpandas-dev
pandas
Python2013–2026
pandas-dev / pandas
pandas
Flexible and powerful data analysis / manipulation library for Python, providing labeled data structures similar to R data.frame objects, statistical functions, and much more
49Distinct people
56Mentions
0Explicit recommenders
50.5KGitHub stars
What to know before exploring
What it is
Flexible and powerful data analysis / manipulation library for Python, providing labeled data structures similar to R data.frame objects, statistical functions, and much more
GitHub
Where it is used
No documented product use was verified in the last source check.
What people are saying
No recommendation phrase was found in the indexed comments.
No classified warning is attached to this page. Read the discussions before treating silence as approval.
Also in these discussions
No other repository is repeatedly linked in the same discussions.
This week on Hacker News
Nov 8–Nov 15 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/15/2026, 5:05:26 PM UTC.
No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.
Mentions over time
HN mentions by year56 dated mentions
Observed mentions: 2013-09-30 22:03:17 UTC to 2026-03-13 23:46:22 UTC.
* An incomplete calendar period is marked with an outlined bar. Its count is not directly comparable with a full period.
Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.
View mention counts and date ranges
| Period | Start (UTC) | End (exclusive, UTC) | Mentions | Submissions | Comments |
|---|---|---|---|---|---|
| 2013 | 2013-01-01 00:00:00 UTC | 2014-01-01 00:00:00 UTC | 1 | 0 | 1 |
| 2014 | 2014-01-01 00:00:00 UTC | 2015-01-01 00:00:00 UTC | 4 | 0 | 4 |
| 2015 | 2015-01-01 00:00:00 UTC | 2016-01-01 00:00:00 UTC | 4 | 0 | 4 |
| 2016 | 2016-01-01 00:00:00 UTC | 2017-01-01 00:00:00 UTC | 3 | 0 | 3 |
| 2017 | 2017-01-01 00:00:00 UTC | 2018-01-01 00:00:00 UTC | 6 | 1 | 5 |
| 2018 | 2018-01-01 00:00:00 UTC | 2019-01-01 00:00:00 UTC | 5 | 2 | 3 |
| 2019 | 2019-01-01 00:00:00 UTC | 2020-01-01 00:00:00 UTC | 2 | 0 | 2 |
| 2020 | 2020-01-01 00:00:00 UTC | 2021-01-01 00:00:00 UTC | 7 | 4 | 3 |
| 2021 | 2021-01-01 00:00:00 UTC | 2022-01-01 00:00:00 UTC | 3 | 0 | 3 |
| 2022 | 2022-01-01 00:00:00 UTC | 2023-01-01 00:00:00 UTC | 0 | 0 | 0 |
| 2023 | 2023-01-01 00:00:00 UTC | 2024-01-01 00:00:00 UTC | 7 | 1 | 6 |
| 2024 | 2024-01-01 00:00:00 UTC | 2025-01-01 00:00:00 UTC | 5 | 1 | 4 |
| 2025 | 2025-01-01 00:00:00 UTC | 2026-01-01 00:00:00 UTC | 4 | 0 | 4 |
| 2026 (partial) | 2026-01-01 00:00:00 UTC | 2027-01-01 00:00:00 UTC | 5 | 2 | 3 |
What Hacker News says
7 sourcesWow! That's a nifty little library. Just looking at the amount of options going into the main parser routine is mind blowing [1].
And the figuring out of encoding is also complex [2].
Once again, this is just to demonstrate that writing your own CSV parser from scratch is a total waste of time. Just use tools provided by your language. Many languages provide native support for CSV parsing. For example see docs for Microsoft C# [3].
----------
[1] https://github.com/pandas-dev/pandas/blob/v1.0.3/pandas/io/p...
[2] https://github.com/pandas-dev/pandas/blob/v1.0.3/pandas/io/p...
[3] https://docs.microsoft.com/en-us/dotnet/csharp/programming-g...
In So you want to write your own CSV code (2014)GitHub commit history attributes squashed commit to committer instead of author
4 points · Open discussionI dare to promote one StackOverflow question [1] about pandas I have tried to investigate and answer [2] half a year ago. And I was rather horrified by its internal complexity after digging into pandas source :)
OP was wondering, why pandas facing a strange overhead after each 100th iteration in some very specific case. There was a proposal about Python's GC, but it was not clear at all.
Finally, I have dived into pandas and found that it has a hard-coded constant == 100 (!) of a number of internal data storage blocks. After reaching this value it runs some consolidation routines [3], and they consume a lot of memory even leading to crash with memory error.
What was much more wondering, is that after changing this constant to some large value (1000000, actually it disables consolidation at all) reduces memory consumption dramatically! Th
Read the rest
is consolidation seems to reduce storage and memory consumption, so I still do not know why the opposite happens and why it works well in all other cases.
[1] https://stackoverflow.com/questions/56690909/python-is-facin...
[2] https://stackoverflow.com/a/56705419/978424
[3] https://github.com/pandas-dev/pandas/blob/761bceb77d44aa63b7...
I've had to dive into the pandas code over the last year for a project [0], and my attitude has shifted dramatically from...
* old attitude: why does pandas have to make things so hard
* new attitude: pandas has a crazy difficult job
I think this is most apparent in the functions that decide what "[d]type" a Block--the most basic thing that stores data in pandas--should be.
https://github.com/pandas-dev/pandas/blob/4edcc5541ff3f6470f...
And then, for the ubiquitous Object dtype, often figure out which of the many possible more specific types to cast it to.
If you think that is easy, ask yourself what this outputs:
import numpy as np
np.array([np.nan, 'a'])
Lo and behold--it produces an array where the np.nan has been converted to the string "nan".
And yet
import pandas as pd
pd.Series([np.nan, "a"])
Knows this, has your b
Read the rest
ack, and does not stringify it.
It also has a pathological fixation on when it tries to convert dtypes, since avoiding all the bad conversion outcomes is a relatively time intensive process (compared to e.g. creating a numpy array).
I realize things could be much easier in pandas user facing interface, but really appreciate the sheer amount of effort that has gone into its dtype wrangling.
[0]: http://github.com/machow/siuba