789 karma · joined October 24, 2012
The linked article talks about Wilhelm von Humboldt's philosophy of education. While I haven't read much into 19th-century German literature, the article seems to suggest that a national education system is foundational in nation building and, possibly on-brand romanticism, that the final goal of education is to produce "independent, critical thinkers".
The same ideals have driven the initial push for public schooling in the United States (which happened at the same time at least in the big East Coast cities). However, with the expansion west, schooling became more of an assimilation instrument, where the preparation of "informed citizens" became more of the goal. This led to public school clashes with established religious schools (mostly Catholic in Chicago and in California), which then resulted in a full separation between public and religious school funding.
The goal of education seemed to have changed with the beginning of the 20th century and the push for universal high school. Powell, Farrar, & Cohen argue in "Shopping Mall High School"[^1] that universal secondary education forced schools to become more “consumer-oriented" by offering classes and activities (i.e., sports) that would keep students in school until 18, while compromising with their original ideals to prepare citizens or critical thinkers.
[^1]: https://en.wikipedia.org/wiki/Shopping_mall_high_school
I really appreciate the API for calling stats functions and retrieving results. This seamless integration was exactly what I was missing.
Regarding my concerns, the project currently gets you through about a first semester's worth of grad-level statistics or quantitative methods. It's sufficient for exploratory descriptive statistics, paired tests, and basic linear regressions. However, this also means it isn't close to the "production-level" statistics required for rigorous research work. At a minimum, it needs to support heteroskedastic-robust standard errors (Huber-White and clustered at a minimum; jackknife and bootstrapped as a non-parametric bonus), multilevel linear models, and generalized linear models (GLMs). I notice there is an open issue for GLM support, though it sounds like that will require a full rewrite of the inference backend. Including marginal estimates following a regression would also be highly useful, especially if GLMs are implemented.
Taking this library from an MVP to a production-ready replacement for SAS, R, or Stata will require significant effort. I am unsure about the market fit for a tool like this; organizations paying for SAS or Stata are unlikely to abandon them for an upstart project, and R has a deeply entrenched ecosystem that is impossible to replace. The situation feels similar to the dynamic between Octave and Matlab, or SageMath and Mathematica. It risks becoming a free alternative used primarily by those who cannot afford the paid products.
As others in the thread have pointed out, this extension includes functionality that is already handled well by other community extensions (like ggsql, stochastic, and read_stat). Because the ultimate goal seems to be providing a SAS-compatible frontend built on DuckDB, which is a huge and exciting undertaking. I wonder if the statistical backend might progress faster by focusing only on tests and regressions. Since it serves as the foundation for the other frontend services, zeroing in on the core stats and relying on the broader DuckDB ecosystem for the rest might make this massive scope a bit more manageable.
If it were a regular 2-lane traffic circle, it wouldn't be too difficult to navigate.
Personally, I find multiple-lane traffic circles (4+) to be more difficult to navigate, mostly when some people in the center lanes pull a fast right turn across all lanes of traffic. Otherwise, much better than traffic lights at keeping traffic moving or first-come, first-served stop intersections when people don't really stop or sit there waving you through.
I think that some of the biggest issues with omitted variables could have been address by using the "new" Stata interaction syntax ("new" in the sense that has been available in Stata since 10 releases ago) instead of rolling their own products and forgetting to include the main effect.
Emailing them privately in advance of the meeting will give them the opportunity to think about your input and, in some cases, reply and engage with you about the policy. It might not change their mind, but it will definitely help them see others' perspectives on their upcoming decision.
By using an LLM to fabricate citations, authors are moving away from this noble pursuit of knowledge built on the "shoulders of giants" and show that behind the curtain output volume is what really matters in modern US research communities.
[Data Preparation] --> [Data Analysis] --> [Result Preparation]
Neither Python or R does a good job at all of these.
The original article seems to focus on challenges in using Python for data preparation/processing, mostly pointing out challenges with Pandas and "raw" Python code for data processing.
This could be solved by switching to something like duckdb and SQL to process data.
As far as data analysis, both Python and R have their own niches, depending on field. Similarly, there are other specialized languages (e.g., SAS, Matlab) that are still used for domain-specific applications.
I personally find result preparation somewhat difficult in both Python and R. Stargazer is ok for exporting regression tables but it's not really that great. Graphing is probably better in R within the ggplot universe (I'm aware of the python port).
We didn't find the frozen DuckLake setup useful for our use case. Mostly because the frozen catalog kind of doesn't make sense with the DuckLake philosophy and the cost-benefit wasn't there over a regular duckdb catalog. It also made making updates cumbersome because you need to pull the DuckLake catalog, commit the changes, and re-upload the catalog (instead of just directly updating the Parquet files). I get that we are missing the time travel part of the DuckLake, but that's not critical for us and if it becomes important, we would just roll out a PostgreSQL database to manage the catalog.
ESTA/ETIAS gets automatically approved within a few minutes of paying for the fee (I guess this is true for 99.999% of applicants).
Very few countries allow people to just show up and cross the border. US citizens had that privilege in a lot of places, but it looks like it’s changing now.
My understanding of the Italian political climate of the 60s, 70s, and 80s is that there were political groups/cells (on both the far right and far left) that organized around violent acts to further their political goals (which involved the eventual authoritarian takeover of the Italian government by either the far right or far left). For example, you can think of the Red Brigades to be akin to the Black Panthers, but with actual terrorism.
In contrast, most political violence in America has been less organized and more individual-driven (e.g., see the Oklahoma City Bombing). For better or worse, the police state in the US has been quite successful in addressing and dispersing political groups that advocate for violence as a viable means for societal change.
While the methods alone cannot fix it all ("You can’t fix by analysis what you bungled by design" [1] after all), it gets somewhat closer to unbiased results.
[1]: https://www.degruyterbrill.com/document/doi/10.4159/97806740...
There are interesting null results that get published and are well known. For example, Card & Kruger (1994) was a null result paper showing that increasing the minimum wage has a null effect on employment rates. This result went against the common assumption that increasing wages will decrease employment at the time.
Other null results are either dirty (e.g., big standard errors) or due to process problems (e.g., experimental failure). These are more difficult to publish because it's difficult to learn anything new from these results.
The challenge is that researchers do not know if they are going to get a "good" null or a "bad" one. Most of the time, you have to invest significant effort and time into a project, only to get a null result at the end. These results are difficult to publish in most cases and can lead to the end of careers if someone is pre-tenure or lead to funding problems for anyone.
At the end of the day, a 3-course sequence in a CTE pathway (which is the CA requirements for a high school CTE certificate in California) doesn't prepare you for a career in the same way as being in journalism class prepares you to be a journalist or being in theater prepares you to be an actor. Students will most likely need to pursue some form of post-secondary training (either through a community college or on-the-job) to become somewhat competent in their field.
Looking at the statistics[1], the US went from a 23.2% college completion rate in 1990 to 39.2% completion rate in 2022, or a 67% increase in college degree completions. If you assume that X in the population is constant over time, mechanically you will need to enroll and graduate students from lower percentiles of X in order to increase the overall college completion rate in the whole population.
This process might be particularly acute at "lower tier" institutions that cannot compete with "top tier" institutions for top students.
[1]: https://nces.ed.gov/programs/digest/d20/tables/dt20_104.20.a...
A Parquet file is a static file that has the whole data associated with a table. You can't insert, update, delete, etc. It's just it. It works ok if you have small tables, but it becomes unwieldy if you need to do whole-table replacements each time your data changes.
Apache Iceberg fixes this problem by adding a metadata layer on top of smaller Parquet files (at a 300,000 ft overview).
This could become a two pronged problem where you have fewer people to provide care while they are trying to implement a new EHR which results in a decrease in productivity because of the need of learning a new system.
It seems to me that going from "F1 and F2 generations of mice respond differently to the smell of acetophenone if their parents were exposed to it" to "well, human trauma is inherited and there isn't anything we can do about violent behavior" is somewhat far-fetched and smells like neo-eugenics.
If you have individual ELOs, you could pre-seed the ranking using averages and then let the algorithm take over.
An interesting consequence about the wearable ban is that they also banned medical electronic devices (e.g., heart monitors or blood sugar monitors) and parents are required to meet with administration to approve their use [1].
[1]: https://greenwichfreepress.com/schools/greenwich-schools-ann...
By the way, not even government agencies have good geo data, even when they should. I needed up to date address information for work, so I bought a map from my local county assessor's office. In my mind, the assessor should have the most recent data on properties, as their main mission is to collect taxes annually. I was wrong. Their data is about 4 to 5 years wrong, with whole "new" subdivisions missing from their inventory. Google Maps kind of has them on the map; I believe that their geolocation data comes from real estate platforms when new houses are on the market. OSM is about 10 years behind in my area. I am submitting edits as I find them.
If someone has a better idea on where to find address data, please let me know.
This is different from qualitative analysis because the data reduction is done “by hand” by the researcher.
The difference between the “automatic”, model-based data reduction in quantitative research and the “subjective” reduction in qualitative research is then amplified when people say that quant is more objective than qual analysis. The discussion, instead, should be about the quality of the work and whether the final conclusions are warranted by the methods instead of the method itself.
The blog post list this as the original Stata code:
> replace event`i’ = 1 if delta_mct`i’ != 0 | spouse_delta_mct`i’ != 0
and this as the correct one:
> replace event`i’ = 1 if (delta_mct`i’ != 0 | spouse_delta_mct`i’ != 0) & delta_mct`i’ != . & spouse_delta_mct`i’ != .
It looks like the authors didn't properly handle missing values in Stata, leading to marking people with missing health information being marked as being "severely ill" instead of being excluded from the analysis.
It's an unfortunate mistake, but it happens.
Let's say that you are a doctor and you live in a town of 20,000 people with one hospital. You are stuck being "loyal" to that hospital unless you move. Now let's say that you live in a metropolis of 2 million people and multiple hospitals. That gives you the option to move between jobs if they become available.
My impression is that cities and towns used to be smaller in the past and commuter culture wasn't as popular as it is now.
I'm ok looking at the past with rose-tinted glasses, but context is also needed to understand the decisions that people made back then.
Let’s say you want to purchase a math tutoring platform. The cost is $30 per student per year. CA schools get about $5 per student per year in lottery funds to purchase instructional materials (shared across all materials). General fund is pretty much all spent on staff/teachers or facilities. One time covid funds are running out. Schools can purchase textbooks at ~$100 per unit and use them for years without subscriptions.
In short, there is no money in public education for these tools.
These platforms look great, but the price is prohibitive. Maybe some basic aid district could pick them up, but those are usually small (I.e., it won’t keep a start up in business).
Note. This is an observation from California. Other states might find education better.
If a paper is broken by design, there isn't much to do after the fact. It's just broken.
The problem is that doing a good RCT takes both time and effort, with the huge risk of having null results, which usually results in a desk rejection from most top journals.
So, you either are a top-fund raising researcher who can both fund multiple RCTs and people to support them, or you just try your best with what you have and hope to squeeze a paper out from you did.
Releasing the data won't really help much if the data generating process is flawed. Sure, other people will be able to run different kind of analyses (e.g., jackknife your standard errors instead of just using a robust correction), but I'm not sure how helpful that will be.
A third issue that I have also encountered is that journal editors have an agenda when putting together an issue, which sometimes overwrites the "quality' of the research with "fit" to the issue. This could lead to "lower quality" articles to be published because they fit the (often unspoken) direction of the journal. Most editors see their role as steering the field towards new directions (a sort of a meta service to the field) and sometimes that comes at the expense of the quality of the work.
It's obviously an issue with point of view, but considering only relative velocity isn't also a solution.
If two spacecrafts are on a close approach trajectory, one of them might need to course correct to fix that. Changing the relative direction might allow them to avoid the collision, but will also change their orbit, leading to other problems down the road (e.g., out of wack apo/periapsis). Fixing the orbit after the close approach will also cost more fuel and/or angular momentum, which is in short supply on most spacecrafts.
I like to think about it like two nascar cars. Sure, if you are driving one, the other cars seem to be almost not moving from your seat, but the spectators see something completely different. Any small change (from the driver's view) in steering can have big effects overall.