Ask HN: Big Data Performance Issues: R vs. Stata?
My question is about the relative performance of R and STATA for logit models. Since I have a large data set, I experience quite slow computation times for my code, in both languages.
-R: I am using tableone() package which isn't particularly fast, I suspect, and I use dplyr and data.table for munging. If I wrote my own function for making tables, do you think it would be any faster? I could also try to utilize data.table for a higher % of the coding process?
-STATA: Do I really need to do my regressions in STATA? I have heard they are much slower in R, especially given that our fixed effects model has around 150 hospital sites to adjust each model for. Ideally, couldn't I do this within R so I don't have to pipe the data from STATA back into R? Or is there an easy way to export STATA output into R to then munge into tables?
Thanks for your help. I am stressing on this project and I want to level up the automation. I just spent 6 hours hand-filling cells for a table and I vowed to never do so again!