Gitoxide: Pure Rust Implementation of Git
github.com
github.com
Programming language flamewar is not interesting, so please don't do it here.
I am one of the devs who got attracted to Rust's advantages over C/C++ (no double free or use after free, guaranteed memory ownership, relatively fearless concurrency) and have been using it increasingly to the point of now professionally working with it.
But I never felt the need to degrade others for not using it. I do have my strong opinions about when and where it should be used but I almost never escalate them to flaming (sadly it occasionally happens even to the best humans).
I'd hate it if such an excellent technology's well-deserved adoption is held back because its core user base is viewed as zealots.
:(
(proof: https://keybase.io/byronbates)
Actual numbers for what's there, pack access and traversal, can be found here: https://github.com/Byron/gitoxide/issues/1 and here: https://github.com/Byron/gitoxide/issues/5
It is my hope that the quality and stability will be high enough to convince people [in the Rust ecosystem] to move away from libgit2 and instead contribute to the maintenance and development of gitoxide, rewarding everyone with better performance and improved usability.
What in your opinion would be a good place for someone to start contributing to this?
My 2 cents. I think rust-analyzer offers a great new dev onboarding experience. issues labelled with has-instructions usually have a a link to the relevant snippet of code in the repo and sometimes even the stub of a failing unit test. While it might be obvious to you, the author, how and where to find the necessary code, it enables a great many people to send patches.
Witness the number of contributors with <10 commits https://github.com/rust-analyzer/rust-analyzer/graphs/contri...
If you want to increase adoption, please borrow as many dev experience tricks from rust-analyzer as possible
I haven't looked at the internals of Git much, but performance becomes an issue with it in this context (we currently use a different vcs), and switching seems impractical at the moment. Issues from local performance (30 seconds or so for git status, git checkout, etc) and clone/fetch speeds.
I'm curious, given your experience on porting, have you noticed areas where we might be able to make improvements that might be easier based on Rust's safety improvements? Such as more threading or other areas that could improve Git for use with very large code bases?
(I plan to review the project as this would be an exciting area to work in to help with productivity in this space)
The same applies when checking for changes on disk - right now git does not do that in parallel. These are low hanging fruit that I plan to pick for my own sake.
There is another contributor who is very interested in increasing pack performance, which will directly impact fetch and pull speeds. Judging from my experience with packs thus far, I believe a lot of options are still to be tapped in that field as well. Rust will open it up for contributions and experiments to a greater audience, so I would hope that this will go way beyond of what I can do.
Even though this project is not very contributor friendly right now, I will be working on improving this so more people can join in (see https://github.com/Byron/gitoxide/issues/8).
Based on my experience, the main value is that it makes this code easily portable.
Meaning, if you can compile rust on your device, you can just run this code without worrying about also installing and appropriately linking the correct version of some C library. It's a huge convenience.
Of course, this only works if all dependencies are also pure Rust. I haven't verified that this is the case here. Here's hoping!
* C is more portable than Rust because there are C compilers available for more platforms than there are Rust compilers for Rust platforms.
* Rust is more portable than C because Rust programs tend to be written in a more platform agnostic way, and are therefore more likely to be able to work on platforms that they weren't originally intended for.
(To make the second point more concrete, as a Windows user, pure Rust programs almost always Just Work for me, but as soon as we get into C dependencies, it becomes way, way way worse.)
Proof: just look at graphics. You just write your graphics code in a platform-independent way and use the correct .BGI file for your graphics hardware.
And it's safe: no pointer arithmetic or void; real array passing. Module system for dependencies. No makefiles needed. No problem of linking a stale object file that didn't get rebuilt due to a missing dependency rule.
Turbo Pascal is the golden future, my friends.
If you’re talking about code portability of pure C, no dependencies, then I agree. Once you start depending on platform features, in my experience, the portability of Rust is superior.
Where is the Rust compiler for a Motorola 68K?
How about PIC24? And if there is ever a Rust compiler for PIC24, what will happen to all that run-time support? How much of it will work? How will you detect what of it works and what doesn't, in order to work around for it in your program?
Someone was working on a 68k port. I am not sure what its status is today.
PIC24 should be totally possible; we have AVR support in nightly now, which is also a 16 bit platform.
Maybe what it really means is that "once you start depending on platform features, the portability of Rust to a handful of mainstream platforms which have all those features in slightly varying flavors is cleaner due to some layering".
You don't have annoyances like, oh, foo_t requires <sys/types.h> on this platform, but on that other one it was already exposed in <stdlib.h>.
In any case. You’re not wrong. We can caveat everything about Rust as “for the platforms it supports”, and in this context it is more enjoyable to work with across those platforms. And for all of my development needs, it targets everything I work with.
C is more portable as a language, but that's never been up for debate.
In C there are also portability libraries. So for instance, suppose you want to do asynchronous I/O with sockets on a few contemporary platforms. You could do all the hard work and make it work with kevent on BSD, epoll_wait there, or I/O completion ports on Windows, with some select and poll based fallback cases.
Or you could just use, whatsitcalled, ... libuv and follow its API. Libuv has the grotty multiple implementations of the same thing in three or four ways, and ifdefs and whatever, so you don't have to.
Then you're into problems like: do you build libuv yourself and ship that? Or do you rely on the upstream one available in some of your target platforms, and which version will that be, and will that cause issues. It's going to be dynamically linked, so users can run with a different version than what you tested with as they update their systems.
But that's just due to having more choices.
If libuv weren't widely adopted, then that would be easier: there is no platform version of libuv. You want it, you build it yourself. You choose the version, pin to that, and you control if/when you update.
All that complexity is from more independent parties being at the table. It's not just you, Developer versus some Mother Ship (language that provides everything, including platform stuff).
We all know that C is portable and has many libraries that support that. The statement is that Rust is in the opinion of many of us that have experience with porting both languages to multiple platforms, that Rust is easier.
mio is the portable library many use in Rust for IO across multiple platforms, it's used by Tokio and other async frameworks to support async-io.
And, yes, if something doesn't exist you must build the support for it. This is true for anything? It feels like you're trying to change the goal posts on the discussion, to be one of library support now. But luckily in Rust, we have bindgen that makes it relatively painless to support C libraries when you want the "easy" path to getting libraries that are implemented there, but not yet in Rust.
It's just that, with the benefit of hindsight, we designed support in three major layers:
* core, which only relies on there being like, a symbol for memcpy
* alloc, which gives you a heap
* std, which gives you operating system-dependent features
Based on this hierarchy, a lot of libraries that are OS dependent may use only core, or core + alloc. And way more crates do this than you might think, and it's pretty easy to move a crate to them if they truly don't need all that libstd has to offer.
If it's pure rust, then I don't have to worry about dynamic or even static linking. I just compile the code for the target and I'm done. Cross compiling becomes simple. Installing becomes simple.
I'm repeating myself a bit here but just to clarify: there's no question that pure C is awesome for portability. And like with Rust it becomes less fun once dynamic linking gets involved.
Now certainly there are platform-specific things with Rust as well, but the defaults are structured in such a way that exposed APIs are more likely than not to be platform independent--you have to go out of your way to break compatibility, rather than it being the default.
And once you get your binary to compile, the static linking means there isn't runtime dependencies that might break either.
The net result is that given a piece of rust code, it is far more likely than its C equivalent to compile on a new platform. And once it does compile, it can be run in any context without concern for differences in the runtime environment. The Rust compiler may target fewer platforms, and therefore be "less portable" in that sense, but if it is a supported backend then it is going to be easier to get the code working.
Both sides are fair points and it depends on a plethora of other factors which should be chosen for any given project.
Take a random, non-trivial C program or library and try to compile it for something that's not a Unix (edit: or cross-compile it). How hard is it to do? Now, do the same with a random Rust crate. It's probably a lot easier.
I think there are two key differences. First, Rust has a standard build system (Cargo) that is dominant, while there are many competing build systems for C. Second, the Rust standard library was designed much more recently, and covers far more of the functionality in today's platforms than the C standard library does. (For example, contrast Rust's OsStr and Path types with the pain of supporting Unicode filenames on both Unix and Windows in C.)
I would guess this will slowly change as LLVM gets more backends, but that's pretty hand-wavy. The other option is we move more towards platform monoculture, which is probably more likely? Not sure.
cough cough Windows cough
Even after years of using git, this is for me a major pain point.
> a simple command-line interface is provided for the most common git operations, optimized for user experience. A simple-git if you so will.
But an explicit non-goal is
> replicate git command functionality perfectly git is git, and there is no reason to not use it. Our path is the one of simplicity to make getting started with git easy. a simple command-line interface is provided for the most common git operations, optimized for user experience. A simple-git if you so will.
So it depends. They may refine some parts of the UX, others may be identical, and other still will be left unimplemented because git still exists.
What you've done is awesome!
What I would love to see is a pure-Rust reproduction of the git commands we know, some simple-git flavor commands, a Git API we can call from Rust, and the ability to add new Rust written git commands. I would love to put my money where my mouth is and volunteer to help, but my job right now is consuming all of my time not spent on family.
Furthermore, git aliases work on your machine. Then you ssh to a server and suddenly `git please-do-my-command-without-many-weird-options` don't work. Yes you can sync them, but not super-easily. Whereas a complete tool you can just distribute onto the machines you work on. Not entirely different, but not the same either.
Please note that everything I name can certainly be fixed by some tool that already exists, and that's great. It's just that I would like to have 'this one tool' that gets most things right and be happy with it alone.
That said, here is my list:
* adding individual files to the staging area/index
* looking at the commit history and copying individual commit hashes
* pulling while assuring no merge commit is created, or: simplified trunk-based development
* resumable clones
* seeing what's going on during all remote operations - here in China GitHub is incredibly slow most of the time and connections can be flaky
The third one is a bit unclear to me:
> looking at the commit history and copying individual commit hashes
What do you mean exactly by that?
It will be nice instead of proclaiming Rust as “C” replacement it should have positioned as “A language complementing C/C++ to write safe code”.
Recently Servo project is abandoned by Mozilla for which Rust is a primary language.
Now need to see, if Rust can fulfils it’s primary objective to write Firefox in Rust, it looks more remote now. So it will take much longer for Rust to become as ubiquitous and important as Go language for systems programming. Probably Microsoft and others might pick to write some core new systems in Rust, instead of just rewriting same working C/C++ code in Rust for marketing purposes.
"Pure" is not some Rust marketing ploy :-)
It's a pretty established term meaning "written only with". I've seen pure Python, pure PHP, pure JavaScript, etc libraries and apps.
It generally means you don't need another compiler to build or you don't need to know another language to contribute or fix things for yourself.
So "Pure Rust" is also marketing when you look at the crates used by this software are not Pure Rust, but rely on unsafe code of C underneath.
Python, PHP do not claim as C/C++ replacement, indeed Python is synonymous with CPython written in C.
In Rust community so far what I noticed is that they try to down-vote anything which shows them that Go language or Swift are more important systems programming language and Rust needs another 10 years to reach to write enough useful software like Go or Swift.
Also in Python look at the example of pure python HTTP server Cheroot [1] and check how many PyPI packages it relies on. Also it can run in CPython, PyPy or Jython or any implementation of Python. Rust does not have anything like this yet so using Pure is misleading given it only has one compiler dependent on LLVM (written in C++) to generate target binary.
If I'm right remember that both your self worth and your employability are ultimately not tied to C. Programmers switching programming languages are not life long miners that have to turn overnight into lawyers. You start building something with Rust and you should be operational quite quickly, most of your skills transfer over.
Rust is anything but easy and simple like Go language.
Please try building something like Kubernetes in Rust instead of Go and you will realize “Rust” is a a complicated language (worse than complex). Rust has a steep learning curve and the standard library is quite limited. So you will need to search and choose from different crates to do eve basic work. Eventually one can be productive in it after 2-3 years of development. Compared to it Swift, Go are much easier to pick up as systems programming language and C++ 20 is advanced enough to write safety critical code. There is a reason why large body of systems programming and software for database,network servers is written in Go language not Rust.
And Swift is only relevant because it's the only modern, officially sanctioned, systems software programming language on 2 platforms: Mac OS and iOS (especially iOS). I know of 0 people who use or have a desire to use it outside of MacOS or iOS.
Time will tell, but Microsoft, Amazon, etc. have adopted Rust. And if you look at what they're doing with it, it is generally much lower level than Go or Swift stuff. It's directly in the C/C++ domain.
Agree time will tell at present Rust is like other niche systems programming language competing with others like Nim and Zig bit more popular due to Firefox being popular. Nowhere close to Go or Swift, may be in 5 years might have something substantial by Amazon and Microsoft available in open source.
> Go has much bigger backing
Go is popular because it’s easy, simple, performant with opinionated design and similar to Python controlled heavily by its creator. It gets the job done and it’s syntax can fit in brain, without fighting the language itself.
It has very good standard library.
It is adopted more outside of Google than inside.
Simplicity is easier said than done and Go language did it quite well. It’s like Apple devices and OS beautiful and simple, which works but some just hate it.
Rust doesn’t aim to be simple and easy, as it makes people in Rust community feel superior to other programmers by virtue of showing it as complex and difficult.
You can see example of this in this thread any comments critical of Rust pointing its shortcomings is downvoted because it’s difficult to digest how the best and safe language for systems programming can have any shortcomings.
I don’t mind working in Rust (indeed tried some simple examples and didn’t find any feature special what I can’t do in other languages including safety). I also learned Haskell (which is more complex than Rust) and Lisp (like it more because of its simplicity).
Rust will never be extremely popular, just because it is more low level. I'd say that for each low level concept you add (pointers/references being on of the big ones), your "market reach" for a programming language goes down by an order of magnitude :-)
Rust sure isn't "simple" for anyone, but if you are a C/C++ developer, I'd definitely expect someone who's been writing C code for more than a few years to be able to pick up basic Rust in a few weeks.
Instead of being threatened by other languages and rationalizing why it's not worth picking up, why not spend a few weeks on it so you can have an informed opinion on why it's good or bad?
No language is perfect, everybody knows that. Rust isn't for every use case just like C, PHP or Python isn't for every use case -- but knowing a little bit about all of them means you'll be a better programmer in the end, because you can recognize what tool fits best for the job.
Also, C++ is a waaay more quirky and complex language. Can't see why C++'s complexity is fine, but Rust's (lower) complexity isn't.
I've been using Rust for 3+ years, and while it has a learning curve wrt lifetimes, it's nowhere near as hard to learn as C++ IMO. I recently tried to pick up C++ (yes, I know, weird to have gone to Rust prior to C++), and I finally just gave up and went back to Rust because nothing works how I would expect it to. There are so many edge cases and weird things you have to know to use C++ effectively.
Also add in the fact that you still need to learn lifetimes in C or C++, except now you are responsible for tracking them rather than the compiler.
Can someone give me an intuition that doesn't clash with the intended meaning like that?
F.ex “effort“ could be quantified as topics in a book to cover before being ready for the next chapter, and “mastery” as simply number of chapters read.
In other words, the issue with your intuition is that the speed at which you go up the curve is constant, but in the saying, you go up the steeper curve much more slowly than you do the slight curve.
Does that help?
>Arguably, the common English use is due to metaphorical interpretation of the curve as a hill to climb. (A steeper hill is initially hard, while a gentle slope is less strainful, though sometimes rather tedious.
So common, that it's basically become the default interpretation. Kind of like "irregardless". I wonder if there is a name for phrases like this that basically mean the opposite of what the words literally say.
At this point, it's clear that you are simply trolling, because Kubernetes is a well known counter example against Go: the lack of features was so blatant they had to design their own generics system as a preprocessing step.
So "Pure C/C++" is also marketing when you look at the libraries used by this software are not Pure C/C++, but rely on unsafe assembly code underneath.
Python, PHP do not claim as assembly replacement, indeed Python is synonymous with CPython which is ultimately written in an assembly language.
Also, C/C++ relies on LLVM, which runs on assembly language to generate the target binary.
Your assertion is not true in C/C++ community, but “Rust is good and C or any other systems programming language is bad” is common assertion in Rust community. Most C/C++ programmers will not mind writing handcrafted assembly. Seen plenty of such examples in game and embedded systems development. C/C++ contrary to your arguments make it very easy to incorporate handcrafted assembly along with C/C++ code, haven’t seen handcrafted assembly code in Rust so can’t say if it’s as easy as C.
Porting code will have some stumbling blocks, but given enough time and some resources those mountains can be crossed and may uncover new trail and then leave the ropes for other climbers.
Personally I am from the camp who likes the simplicity of lisp. That’s one of the reason in spite of learning some Haskell to learn about FP, didn’t continue long although I like Haskell’s brevity as in “Brevity is the soul of wit”. Haskell really makes it easy to refactor the code, but given large surface area makes it hard to learn and develop. Rust felt the same way although recently saw it becoming more approachable and easy.
I was pondering recently if built a modern lisp machines, how good it will be. A hardware directly controlled from lisp instead of C.
This is not Rust's position on safety. Rust's position is that you should be able to isolate unsafe parts of your program. And we try to give you tools to make them as small as possible. But unsafe stuff was deliberately included in Rust, on purpose, as a tool to be used.
I also wonder what a modern lisp machine might look like. Oh well, I don't suspect that will ever happen.
This was what Mozilla hoped to do with it. But the teams at Google, Facebook, Amazon, Microsoft, Dropbox, Cloudlfare, fastly and others using Rust have different objectives. At this point, Firefox's success and Rust's success are decoupled.
'Pure Rust' counts for little if the code overuses Rust's unsafe features.
edit This was a sincere question. I'm glad the answer turned out to be that there's very little of it.
There was an HN discussion a few months back on a Rust program that overused the unsafe features, to the point it may as well have been written in C. Sure enough, it turned out to be full of nasty bugs.
Mildly disagree. Even if half the code base is unsafe, that's better than a code base where everything is unsafe. Now I only have to audit half the code base for crazy shit, which is about half the effort. A stronger type system means that even in unsafe land, we can still encode some invariants better.
I disagree with this notion that "if you're going to have bugs anyway, what's the point of Rust". Our goal isn't to eliminate every bug in the world, it's to have fewer bugs today than we did yesterday. Rust, even Rust with several unsafes thrown in, helps us achieve that goal.
[citation needed]. And it would be hard to find a citation because most Rust codebases don't have much unsafe. The repo you're thinking of was an anomaly and has since improved a lot.
Well, this can be related to an actual real-world concept, that is "programming in a certain language with the syntax/runtime of another".
In my experience, all the times I've seen this happening, the project was just terribly written.
I program in Rust, and if I happened to put an excessive¹ amount of unsafe code in Rust, it means that simply, I'm not programming in Rust. In my opinion, in real world this phenomenon happens for beginners, lazy programmers, or bad programmers, especially for Rust, because it has a steep learning curve, and it requires an effort, when designing a project, that other languages don't require.
So there is a point, although theoretical, that a good programmer in an unsafe language, can produce a more stable project than a bad programmer in a safe language.
With that in mind, I still wish as others to see unsafe languages disappear :-)
¹="excessive" is of course subjective.
Such a downvote heavy community. Plaudits to all involved.
A program can qualify as pure Rust without making use of the safety advantages of the Rust language. It's not unreasonable to wonder how much unsafe code there is in a Rust codebase. It matters.
Rust is better for having unsafe features, but it opens the door to their overuse, and this shouldn't be ignored.
In the SPARK Ada world, there are 'assurance levels' that offer different levels of guarantees. [0] At the 'Silver' level and above, there is a guaranteed absence of runtime errors (such as divide-by-zero, or out-of-bounds array access). Rust doesn't have such a scheme. It seems reasonable to wonder about overuse of unsafe features (as fuzzy as that is), as a very rough approximation.
[0] https://blog.adacore.com/from-ada-to-platinum-spark-a-case-s...
There are reasons to prefer a pure-rust code, even if the whole crate was unsafe.
Again, plaudits to all involved.
---
FTI [0]:
> In 2007, Jeff Atwood made the quote that was popularly referred to as Atwood’s Law:[5] >> “Any application that can be written in JavaScript, will eventually be written in JavaScript.”
Any improvement in this area would need to inhere in library-level facilities constructed for the purpose. What are they?
You can still swallow errors if you want to but you have to manually write unwrap() so it's much more visible.