maxbrunsfeld

The GIF is just intended to represent an idea - that parsing (for the purpose of syntax highlighting and other code analysis) can no longer impact the app's frame rate, because it has been offloaded onto a background thread. The actual operations depicted in the recording are not a performance breakthrough.

Animals don't raise other animals on cruel factory farms.

There's a button to toggle whether these lines are shown or hidden. Implementing this button's behavior using JavaScript seems perfectly reasonable to me.

GitHub has helped to maintain git itself for years, in addition to creating and open-sourcing libgit2, a massive engineering effort, which ironically, GitLab is built on.

I’ve locked a helmet up with my bike all my life, in several different cities; It’s a complete non issue to me. What is the concern; that someone will use a blade to cut the helmet strap so that they can steal your broken helmet? It’s effortless to leave your helmet attached to your bike.

Edit - I’m not saying I support laws enforcing helmets for adults; I just don’t think that helmet theft is a big concern.

The protocol isn't coupled to the DOM or Electron in any way. The one thing that makes it easier to implement in electron is that it uses the WebRTC standard for peer to peer connections.

Do you remember if there was an issue opened for this on the Atom repo? I haven't heard of this problem.

Thanks! Yeah all of the CoffeeScript in atom/atom should be gone in a few months probably. We use plain JS now.

It'll probably take a while before there's no more CoffeeScript in the entire Atom org. We're gradually converting the code to JS as we come upon it for other reasons.

I really want to create a public-facing roadmap that's specific to this issue. Unfortunately, our resources are limited so we often don't focus enough on blogging/publicizing our planning... but in the meantime, here's something of a brain dump:

In terms of our actual data structures and algorithms, we're already starting to be in really good shape. We've dropped a number of components of our core TextBuffer to C++, ensured that most of our algorithms scale logarithmically with file size, cursor count, etc, and made use of native threading for important operations.

1. The one remaining structure that we need to drop to C++ is what we call the 'display index' - the thing that stores the locations of things like folds and soft wraps. Once we do that, opening large files (which is already reasonably fast) will be like butter.

2. Our find-and-replace is already pretty good - you can type without almost any lag even when we're storing the locations of millions of search results. But now that we have the ability to easily use background threads, there are some easy optimizations we could do there. The search results could really update instantaneously, we no longer need to wait until you pause when typing in the search box.

3. We have in the works a major change to our syntax highlighting system using my incremental parsing library Tree-sitter. Once this lands, it should eliminate any perceived latency in syntax highlighting (as well as enable a host of great syntax-related improvements).

4. Atom still uses synchronous IO (which blocks the UI thread) in some places. This is because it was created before GitHub created Electron, so node APIs were not available from the outset. Many of these have been eliminated, but there are several Git-related code paths that we still have not updated. This probably kills the experience of editing on remote drives like SSHFS for some users. We need to revisit these code paths.

You really aren't. Atom runs fine on a Macbook Air. There are definitely performance issues, but we're addressing them; it's just that our team is small and our initial goal at launch was to produce the most hackable text editor possible.

Sorry to hear that. We’re improving Atom’s efficiency all the time, but there are definitely areas that we just haven’t gotten to yet.

Thanks for the really concise example.

We are already able to run native code in Atom via node's native extensions, so we have no reason to wait for WebAssembly.

The only benefit of WebAssembly would be if we wanted to port Atom to work in a regular web browser. We can even already do that without WebAssembly by compiling the superstring library to JS with emscripten. This works today and benchmarks show that it's already quite fast, but I think WebAssembly will make it even more efficient.

I don’t know how you could read this post and not get this: The Atom team is quite capable of writing C++. It’s not painful at all. I love writing low level data structures in C++.

It just wouldn’t be an improvement for the vast majority of our code, because JavaScript already runs very fast thanks to V8, and using JS allows for excellent customizability by our users.

PCRE's partial match[0] API allows it to be used on non-contiguous data structures. We use this feature in Atom[1] to allow for fast regex search in a background thread.

[0] http://www.pcre.org/original/doc/html/pcrepartial.html#SEC2

[1] https://github.com/atom/superstring/blob/ed57b08a74220dd33e4...

Atom uses a piece-table-inspired data structure to represent text[0]. We store the file's original contents in a single contiguous buffer. All of the user's unsaved edits are then stored in a separate mutable structure called a Patch, which we represent as a splay tree. When we need to read from the buffer in a background thread, we can 'freeze' the patch, and store any subsequent writes in a new patch which is layered on top of the previous one.

Even though we do load the entire file into memory (as opposed to mmap-ing the file), the piece table design is still very useful. It makes it very cheap to compute the buffer's current unsaved change set, which is a value that we periodically serialize for crash recovery purposes (similar to vim's `.swp` files).

It's also just a very compact way to store a large chunk of text, which is good from a cache-locality and memory usage perspective.

[0] https://github.com/atom/superstring/blob/master/src/core/tex...

That's one good reason that you might choose to use Electron, but it's not the main reason that Atom uses it.

One of the main goals of Atom is to be the most hackable text editor ever. Given that goal, Electron is the ideal platform for the app. Thanks to node's native module system, we can cleanly drop any component we want down to C++, while keeping our main application logic written in the most widely-known (and fastest-executing, thanks to V8) scripting language in existence. I'm happy with this architecture as opposed to starting in C++ and then bolting on some specific, limited scripting APIs.

To be sure, Atom has performance issues that remain, but we're making steady progress on them, and Electron isn't standing in our way at all.

Atom has always had some C++ components, but lately we have moved more core data structures to C++. This PR, which re-implements the text-buffer in C++, also landed in the latest beta.

https://github.com/atom/atom/pull/14435

FWIW, I'm developing a library based on the technique outlined in this paper (and others by Time Wagner). Like the original paper, it uses LR(1) (and GLR), not LL(k).

The library itself is here: https://github.com/tree-sitter/tree-sitter

and here are some existing grammars:

  * https://github.com/tree-sitter/tree-sitter-javascript
  * https://github.com/tree-sitter/tree-sitter-go
  * https://github.com/tree-sitter/tree-sitter-ruby

This is a parser generator that I'm working on:

https://github.com/tree-sitter/tree-sitter

It produces concrete syntax trees that can be queried by line or character index. The library is specifically focused on incremental parsing, for use in a text editor, but it also works fine for normal 'batch' parsing workloads. It has a simple C API that you should be able to use from any language. Currently, there are bindings to JavaScript and Haskell. Here are some example grammars:

* C - https://github.com/tree-sitter/tree-sitter-c

* JavaScript - https://github.com/tree-sitter/tree-sitter-javascript

* Go - https://github.com/tree-sitter/tree-sitter-go

* Ruby - https://github.com/tree-sitter/tree-sitter-ruby

* Python - https://github.com/tree-sitter/tree-sitter-python

How would a gap buffer handle distributed editing operations like search-and-replace, or multi-cursor typing? The data structure seems optimized for editing in one place at a time.

Is this on 1.14-beta0? If so, please open an issue. We now do a lot of text layout computation lazily, but currently, it's very easy for a third-party package to accidentally force the entire text layout to be computed immediately via an API call.

I'm curious about why higher-kinded types are needed for this. I'd have thought that it would be possible to create these kinds of traits in rust's current type system. Is there someplace I could read more explanation on this?

> CYK and Early parsers are both actually simpler than parsing with derivatives, once you've thrown this paper's optimizations in.

Yeah, GLR is very elegant as well; I'm a little puzzled by this paper's initial claim (and similar claims in the Yacc is Dead paper):

> Current algorithms for context-free parsing inflict a trade-off between ease of understanding, ease of implementation, theoretical complexity, and practical performance. No algorithm achieves all of these properties simultaneously.

GLR is has good performance and isn't much harder to understand than LR, which is a very elegant theory. So I assume that 'ease of implementation' is the property the authors think GLR lacks. But ease is a little hard to measure, until you have an equally fast and production-ready implementation of both algorithms.

Pausing is very straightforward with bottom up algorithms like GLR. The entire state of the parse is stored in an explicit stack data structure, as opposed to implicitly in the program's call stack as with top-down algorithms. So resuming the parse is as simple as resuming the parse loop using a previously-stored stack.

I'm curious about the source of the slowness of the IDL operations. Is it specific to the DOM, or is he referring to overhead that exists for any bindings between JS and C++?

This looks great. Bookmarked for when I get around to learning Rust.

> one thing that struck me is that markers can't be zero-width. Well, they can but they won't show up. I'm wondering if this is related to the technique mentioned here

Thanks for bringing this up. It's not a fundamental limitation at all. Highlights whose markers are zero-width are deliberately filtered out here[1]. This might not be the right move. If you've found it inconvenient, would you mind opening an issue on atom/atom, and describing your use case?

[1] https://github.com/atom/atom/blob/ebc5758d79e421f61f2b6669a8...

The best tool for that (and any profiling in Atom) is chromium's built-in profiler. See the flame graphs in the blog post.

Packages can definitely have a significant impact on Atom's performance. You can observe all kinds of events, and binding a slow handler to certain events like text changes and cursor movements can make the editor feel sluggish.

It's important for Atom's extensibility that these kinds of events are provided, because it allows many major editor features to be implemented as separate packages, but it means that naively-implemented packages can really slow things down.

Many Atom packages are pretty new, and their authors may not have put a lot of effort into optimization yet. In the past few months, the team has put a lot of effort into stabilizing and solidifying our APIs. We're hoping that now that the APIs are stable, the package ecosystem will really start to mature.