Rsuite – R development and data science platform
github.com
github.com
R by itself is a mess, and I don’t think I have to say much about that. R community is big and that’s good and bad. It’s good because amazing people are developing amazing packages for it. It’s bad because there is a lot of bad packages. It’s a lot like JavaScript community. I have a feeling the community has started to reward “having a package”, and everyone has a package.
Besides the quality of R packages and R being a strange programming language, R gets the job done. However, if your job is anything beyond some statistics and data processing, then good luck. I’m not saying that you cannot achieve what you want to achieve using R, however, good luck reading R code. I found it extremely hard to read R codes and so far 90% of codes that I’ve encountered have little to no comments.
This isn't specific to R or RStudio. Start running a slow process in your Python IDE of choice, and while it's running try to execute df.head() to view some data frame - you won't be able to see it regardless of the language or IDE (and for a good reason).
Now, if I want to address your comment. I don’t think “this was the way things have been done here, let’s do it like that from now on” is a good approach. If there is a better way, even if it’s more complicated, at least it has to be tried and tested. If a more “sophisticated” tool proves to be useful, let it be. Not everyone in a team has to use the same set of tools, and if they see the benefit in something more “sophisticated”, they might want to try it, even if it was not explored before. People can learn, there is always a better way/tool and one tool cannot do it all.
I'm probably an outlier, but I have to say that the language itself is one of my favorite things about R.
Vector based, super powerful indexing of vectors, functional programming basics, lazy parameter evaluation, super convenient parameter matching and defaults, all these things make it super productive for me and let me deal with data far better than other languages. Matlab is similar in its ability to deal with data, but that's a language that feels far clunkier to me. Python has caught up with some of its packages, but it definitely feels bolted on instead of native to the language.
This is what Julia aims to solve.
Also, the author of R language said the performance of R is sub-optimal. In his own words: https://www.stat.auckland.ac.nz/~ihaka/downloads/JSM-2010.pd...
Currently I am using PyCharm with R plugin that JetBrains released very recently. R Studio is very slow and buggy.
Chapters two and three are the lesson to not use procedural language fundamentals if you want performance, and to instead use functional equivalents. Not exactly a language quirk.
If somebody has only ever used 2000s-era Java and C# and those types of languages, functional style programming will be a strange beast. But python has enough functional style things such as lost comprehensions, and I hear Java and C++ have gained functional style programming too, so in this day and age I’m not sure that functional style programming should be considered quirky.
The runtime performance is all due to the current implementation, not the language itself. I think JavaScript has far more quirks, it then it was also invented in an insanely short amount of time so that’s not too surprising.
That's from 2010, even before JIT compilation in R became a thing. And since then, much has been done in terms of the computational performance of R.
Yes the language itself has problems. It's 1-indexed for God's sake! But if you stick to what it's good at (dplyr, ggplot2), you can get a lot of mileage. What Linus Torvalds said about C++ programmers probably applies doubly to R programmers - so yea comments are going to be sparse. And if you venture far from its core competence of data, you're gonna have a bad time. But overall Hadly Wickham and the tidyverse are driving the ecosystem forward, and R has found a great niche between python scripting and Excel/Matlab
Again, I’m not saying that it will happen, but it could. Dependent of R on third party packages and a company to push it forward it’s not necessarily a good thing.
P.S. Python IDEs are crappy too!
Erm, not at all. You don't need a hundred packages to run a trivial application. Even R-base is reasonably powerful, and it goes to a complete different level once you use tidyverse as a layer to code everything for R.
https://blog.rstudio.com/2019/03/14/rstudio-1-2-jobs/
Also not sure you can criticize a whole language because some random code you read doesn’t have comments.
My feeling is that R is great for anything that doesn’t need to be operationalized into production (monitoring, security, logging, scaling, performance, etc). There are so many good ML/stats libraries in R and most books seem to use R (when written by academics) but it feels like these people have never had to put anything into production.
As a general rule of thumb, if something needs real time predictions or I need deep learning libraries, I use Python. R is for anything else.
Python is single threaded as well. Like Python, R can be made multi threaded, and like Python, R can be productionized without having to convert it into another language.
One possible implementation is a pool of R workers. Each request calls an R worker. So if your pool is 100 and you get 20 requests from 20 different users at once, all 20 will be ran simultaneously. Likewise, many tasks can and should be cached. Consider MemcacheD or similar.
Moreover after seeing what my colleagues publish as scientific R programs, I came to believe that science itself is bottlenecked by the large scale adoption by R and the sloppy, inconsitent and bug-infested programming practice that it encourages.
R does a few things well - cross-platform, plotting works on all platforms, packaging works well. But for actually programming it is atrocious.
I thought that was what everyone was using it for. What are people using it for?
Anyway, I think it's better to make use of a small, commonly used, and well tested library instead of reinventing the wheel again and again. Libraries are not one of my concerns I have with R.
now what if I told you that the majority (perhaps all) of these libraries you loaded and are needed to run the complex analyses in life sciences were all developed by people who are oblivious to proper software engineering. These were never meant to be used the way they are used - expose myriad of global variable names, methods etc.
You say you are using a small well-tested library with R, sure - but that is not what happens in science and for those that see what is going on, we know we're completely FKDd
The tragedy is that we cannot cure cancer as long we try to do it with R - and R is not going anywhere ...
I don't know a single package that exposes global variables. I only know packages that expose certain functions and provide some configuration via options. In R, there are no really private variables though. If you want, you can access anything you want. Since everthing is packaged in its own environment, I don't see that as a problem though since the global namespace doesn't get polluted. It won't make people with a background in one of the more modern OOP languages happy but that's probably not that important.
But R also makes a lot of the tasks you do in data science far easier than it would be in a 'real language'.
Though, to be fair, I like R. It has good plotting libraries, and as much as it gets bashed I like RStudio too. In comparison, Raku's ecosystem is brand new.
# cat /tmp/test.R
x <- 1:10
y <- 1:20
plot(x, y)
# R --quiet
> source("/tmp/test.R")
Error in xy.coords(x, y, xlabel, ylabel, log) :
'x' and 'y' lengths differ
> traceback()
8: stop("'x' and 'y' lengths differ")
7: xy.coords(x, y, xlabel, ylabel, log)
6: plot.default(x, y)
5: plot(x, y) at test.R#3
4: eval(ei, envir)
3: eval(ei, envir)
2: withVisible(eval(ei, envir))
1: source("/tmp/test.R")
If you mean the lines in the packages you used, I think you'll see the lines if you build them with keep.source=TRUE R -d gdb
that's it! options(error=recover)Sure, most of the matrix operations and tight loops are implemented in FORTRAN or C++, but that's for performance reasons. The same would happen with Python, and I don't think it would be fair to call Python a scripting language either; it also has a ton of Lisp like qualities.
As for the "easy iterop with other languages": [1]
So, how is R different from Python?
Also, it's not true that R wasn't designed for writing software. Even a critical "pamphlet" by Pat Burns [2] states otherwise. For writing statistical software, yes.
--
[1] https://wiki.python.org/moin/IntegratingPythonWithOtherLangu... [2] https://www.burns-stat.com/pages/Present/infernoishR_annotat...
But don’t worry about R bringing science down - the scientific community can also write terrible python code.
What would the others be? Python is one, I guess.
Also potentially PHP?
I have no opinion on JavaScript.
I guess some people may include Perl. Others won't, either because they don't think it's ill-designed or they don't think it's popular :-)
My experience is that people who make bad R programs also make bad python programs. I don't think you should blame a tool for issues caused by the programmer.
there is no escape.
When I said "colleagues" I really meant the entire scientific field runs on untold lines of buggy R code, so obtuse, so cryptic, that the task of debugging or even tracing what is going on is practically impossible. And you can't debug it because it is this awful R code everywhere! And when the code breaks it does not break like normal programming language do, with an error or exception or even a stack dump. No! Most of the time your R code will just start silently doing the wrong thing.
>try loading up any scientific package and you'll see how, in turn loads up other and other packages, in bioinformatics you can easily add up to dozens if not a hundred of dependencies
That's not necessarily a bad thing unless you're trying to run R on embedded or some other constrained environment.
>When I said "colleagues" I really meant the entire scientific field runs on untold lines of buggy R code, so obtuse, so cryptic
I can't recall the last time I've bumped into a bug in an R library. I'm sure they exist but thankfully the ecosystem is quite stable.
>that the task of debugging or even tracing what is going on is practically impossible.
Debugging in R is easier than most languages. I'm unsure where you're getting your facts from.
>And when the code breaks it does not break like normal programming language do, with an error or exception or even a stack dump. No! Most of the time your R code will just start silently doing the wrong thing.
It's no worse than Python in this regard. R isn't particularly bad in this area, but it's certainly no C++.
I'm going back to guessing it's because R is FPP. That's R's dirtiest and most offensive part to the uninitiated.
None of this makes R a bad language, in my opinion. R is far from the only language with surprising edge cases like this. People say that R is designed for statistical analysis more than general programming, but I don't think that's exactly true either. Certainly it excels in writing code for statistical analysis, but I've used R a lot more than that, and I plan to continue. It's a perfectly fine general-use scripting language.
I think the real reason R gets such a bad reputation is that a lot of people writing and publishing R code aren't programmers by trade. And you know what? That's fine. Because I'd much rather work in a community that values and celebrates the publishing of code than one that shames people for releasing their code because it's "not good enough".
The vast majority of packages are written by somebody taking their interactive session and tidying it up with some functions and tests and then publishing it. But going through and weeding out all these "broken by default for edge case" aspects is a nightmare.
https://www.talyarkoni.org/blog/2012/06/08/r-the-master-trol...
https://www.burns-stat.com/pages/Tutor/R_inferno.pdf
and many others.
I have come to believe that the only people that think R is ok are those that are either:
- beginners that just passed the newbie state, learned a few tricks and feel empowered
- actual experts - that fully understand the minute details of the implementation and data models
I have been using R on and off for a decade, as soon as I stop using for a few months getting back is like a tar pit where I am continuously caught off guard by the myriad of ridiculous problems. Paradoxically as you get better with R your errors start becoming more dangerous, your code starts silently doing the wrong things.
R is unlike any other programming language that I have used before (also on an off) from Perl, C to Python and Java. None of these programming languages have such in incredibly obtuse and illogical and trippy design.
You can virtually say the same thing for every programming language that is made to be easy to learn by hiding complexity, like Python or Ruby.
I prefer using Rust rather than paying hundreds of thousands of dollars on a static analysis tool for C++.
Also, to quote a comment[0] of yours from 2 days ago:
What you are saying is that since you prefer something everyone should do the same.
And what you prefer is the correct choice for everyone ...
It's not perfect, but I'm struggling at finding bad things to say about it.
In this post, I make a recommendation of staying away from R. It is not even a remotely similar context.
The thing is, it's easy to write bug-ridden sloppy code in any language. Bemoaning R as a language because of these flaws, occurring due to rapid adoption, ignores the reasons why R has seen wide-scale adoption.
R has had an extreme democratizing effect on access to tools that facilitate data science. Previously, tools for data science were either massively expensive or had a prohibitively high price tag attached.
This means that many non-programmers are coming to R, and I maintain that the problems the parent post sees with R stem from that fact. As a result, any language that achieved that sort of layman (to programming) appeal would have the exact same bug or sloppy code fallout. That you cannot separate the momentum that led to such an accessible tool without have the same consequences. Rather than demonize the tool for this, we should recognize the positive dynamic at play and simply help guide users to better practices or improvements that would fix the issues.
There are many books that cover how to develop in R in detail, and they are no less thorough than treatments of the subject in other languages (e.g. Hadley's books are as good as any I've read for python).
Many issues around inconsistency, etc, in language design (mostly how base functions / data types behave) have very clean, consistent implementations in libraries like rlang.
The main differences I see when comparing R vs python package code, that affect style are...
1. Most R operations are immutable.
2. R often uses single dispatch, rather than putting methods on a class object.
3. In R, vectorised behavior is often the norm.
4. R functions can choose to use lazy evaluation (it usually very clear when this happens in e.g. tidyverse packages).
These issues are covered in detail in books like Hadley's Advanced R.
My hat off to him though!
As for the language ... consider this: lapply(), sapply(), tapply(), vapply() each does something different. The language allows two kinds of assignment operators even: a=1 or a <- 1 that are "almost" identical ... good luck, here is a language there are two ways to even assign a value to a name.
three ways if you count a <<- 1 for writing in global variables from inside functions (of course, not a recommended practice...)
I don't see what you point is though. So there are several ways to do the same thing in a language, so it's bad? Bad in what way?
As for the lapply, sapply, tapply, mapply, it's very well documented as to when and where you should use them. Sapply applies only on a single vector, and for generalization on larger data structures you use the other "applys". Nothing very hard to comprehend, and this is well explained in the official docs.
The <- assignment is normal for functional programming languages. F#, OCaml, S, and more use this operator. This is because the arrow key used to be a physical key on keyboards back in the 70s when FPP was popular and brand new.
The = sign (function assignment operator) is function level scope and <- (assignment operator) is top level scope.
eg:
median(x = 1:10)
x ## Error object 'x' not found
median(y <- 1:10)
y ## [1] 5.5
So therefor, x <- 1:10
median(x)
is equivalent to median(x <- 1:10)
It's a convenient feature the language supports. The alternative is how Python does while loops. If anything, R comes out above in this regard.edit: Python has the := operator which functions the same way <- does in R. I guess Python is catching up on this one.
eg (Python):
env_base = os.environ.get("PYTHONUSERBASE", None)
if env_base:
return env_base
vs if env_base := os.environ.get("PYTHONUSERBASE", None):
return env_baseThere are many people in the R community working on this together (e.g. Jenny Bryan, Charlotte, etc).
> lapply(), sapply(), tapply(), vapply() each does something different.
The apply situation has been standardized through the purrr lib and dplyr for a long time. They are base library functions that aren't mandatory.
> two kinds of assignment operators even
Consider the custom of using <-. It reduces the kinds of assignment operators to 1. Similar to avoiding from lib import * in python. You can do it, but there are community standards against it.
Yet R is a lot more expressive than Python + Pandas for data related applications. It was never made as a universal language to develop any kind of applications, but it's pretty good at what it does with data manipulation.
I really hope you are not using R for anything outside data science, physics, or other analysis. It was developed to do these things, not 'actual programming', which I imagine you define as creating some framework or application.
Most of the people that don't like R seem to want to use it outside of its use cases, and get frustrated when they fail.
Took a quick look at the docs. If you're looking for dependency management there's renv[1] and you can (obviously) use git for source control. If you actually have enterprise use cases for library curation or air-gapped deployments, you can check out RStudio Package Manager[2]. Among other things, it provides precompiled binaries for packages, which Rsuite doesn't improve on, per docs[3]:
> Now you are ready to install dependencies. Beware that it will take a lot of time because of compilation. You install dependencies with the following command:
[1]: https://github.com/rstudio/renv
[2]: https://rstudio.com/products/package-manager/
[3]: https://rsuite.io/RSuite_Tutorial.php?article=rsuite_binary_...
" - Open source with Enteprise [sic] support.
- Designed to separate..."
The very first bullet point has a typo so maybe this isn't very mature yet?
Check docs (https://rsuite.io/RSuite_Tutorial.php) and examples (e.g.https://github.com/WLOGSolutions/RSuite-examples )
R Suite gives answers to the following challenges for any R based software and data science solution:
- Isolated and reproducible projects with controlled dependencies and configuration.
- Separation of business, infrastructural and domain logic.
- Package based solution development.
- Management of custom CRAN-alike repositories.
- Automation of deployment package preparation.
- Flawless integration with Docker.
- Development process integrated with version control system (currently git and svn).
- Working in internetless environments.
>isolated and reproducible projects with controlled dependencies and configuration.
... that would be huge. Sticking it in docker containers is also a decent idea.
Thanks for writing it, and pay no mind to ding dongs on here who can't be bothered to learn the language and its tooling, but sure do have an opinion on the topic.
I have liked R because I use it simply, inside emacs org-mode code source blocks which use either R to generate plots or gnuplot. Based on comments, now I am afraid I will reach some ceiling in R. What else is there? Octave? Sage? Julia?
The R packages for these domains are one of the best I've seen.
As for R in production, I would wrap it using https://www.rplumber.io/.
What do you think of the new renv package?
We started with reproducible project definition. Then we implemented rsuite to help manage the project. It includes dependency management which is what renv solve. What is the biggest difference is that our project consists of possibly many pkgs that are local to it. This allows you to create complex solutions. Moreover deployment PKG is zip file and to use it you only need r. No PKG installation on prod.
[1] https://github.com/github/semantic/issues/382#issuecomment-5...