Advanced techniques for reducing Emacs startup time
blog.d46.us
blog.d46.us
Apparently they are getting close: https://github.com/jwiegley/use-package/issues/282
Until then, I have this in my init.el to install it if necessary:
;; Install use-package because that installs everything else
(unless (package-installed-p 'use-package)
(package-refresh-contents)
(package-install 'use-package))
(require 'use-package) NOTE: use-package is not a package manager! Although use-package does have the useful capability to interface with package managers (see below), its primary purpose is for the configuration and loading of packages.
It can be configured to install things (by using the builtin package.el): https://jwiegley.github.io/use-package/keywords/#ensure-pin
> You can use use-package to load packages from ELPA with package.el. This is particularly useful if you share your .emacs among several machines; the relevant packages are downloaded automatically once declared in your .emacs. The :ensure keyword causes the package(s) to be installed automatically if not already present on your system (set (setq use-package-always-ensure t) if you wish this behavior to be global for all packages)
So if all you need is package.el (and there's a good chance that's the case), you can just
(setq use-package-always-ensure t)
I'm addicted to the flexibility, but I dream of something that offers the same level of flexibility in a more robust ecosystem (e.g. statically typed, statically scoped, what else?). (Is such a thing even possible? I don't know!)
Speak for yourself. Just last weekend I had a few hours with little to do, and actually wanted to do some yak shaving on my Emacs config just for the hell of it. but it works too well and I have everything the way I like it! My yak is f*kin' bald, man!
Not only have I already worked out all the little bugs that annoy me, but I've added all the features I can think of that I want!
Here's my list of problems to solve on a rainy day†:
;; * kill-word-sp cancels ioccur mode for some reason
;; * after running ipython for a while, all terminal commands hang waiting for output
;; * iedit substition starts just corrupting shit (e.g., try replacing \t with spaces)
;; * "replace-match-maybe-edit: Match data clobbered by buffer modification hooks"
;; * delete-horizontal-space doesn't work w/ iedit mode
;; * vc-annotate doesn't properly restore window configuration
;; * the popup used by dumb-jump breaks when line wrapping is enabled and it's used on a wrapped line
;; * sometimes trying to launch a new shell just kills the old shell
;; * godef doesn't work when narrowed
;; * emacs stopped automatically recompiling .emacs wtf
;; * grep-ed while buffer is in loccur mode fucks things up
;; * undo in loccur mode messes things up
;; * visual-line-mode jumps around on very long lines (like paragraph long)
;; * parens don't overwrite properly in go-mode
;; * «Error running timer `auto-revert-buffers': (wrong-type-argument number-or-marker-p nil)»
;; * shell doesn't save history a lot of the time
;; * piece of shit grep edit silently fails on spaces on file names
;; * code-wrap is stupid with long separator lines (like "//---" for a long time)
;; - sadly might be a visual line mode bug, actually...
;; * line highlight doesn't work after jumping to a definition
;; * highlight line mode doesn't work on most recent line in shell
;; * tendency to identify buffers by name causes bugs when renaming buffers ("stringly typed")
;; * subword mode doesn't think DEBUG_MEGA_GOLD is three words (in a .go file)
;; * if a shell command tries to open a file in emacs, and the active window is the minibuffer, the file will open into the minibuffer
† To be fair, not all bugs -- some are just configuration issues I haven't figured out yet.
Change to follow XDG specification and ~/.config/emacs having precedence?
It's been a while since I wrote that note to myself, but I somehow had .emacs setup so it'd get recompiled automatically to byte code on changes (which helps startup speed), and somehow whatever I setup at some point stopped working. (Which resulted in my .emacs changes being silently ignored, as stale .elc was used -- that was confusing to figure out!)
For now I just gave up and stopped using the byte-compiled version, sometimes life is too short, even for an emacs user!
(Though in general anyone who chooses to use emacs must not think life is TOO short...)
There's no difference between an advanced emacs config and a software project.
If you don't test or incubate features long enough before your workflow depends on them you'll find yourself building atop more bugs.
I find tagging releases of my emacs config useful for this.
E.g., there's a mode that highlights the current line. And there's a mode that let's you run a shell inside a buffer.
And for some reason, the current line highlight doesn't work on the most-recent (waiting for input) line.
OK, actually looking at this a little more closely, I think it's because shell mode does some sort of thing where it redefines the "start" of the line to not include the shell prompt, and I guess whatever is doing the line highlighting is using that same notion of "start" of line. So on a line that just contains the prompt, the line highlighter thinks the line is zero length, and highlights nothing.
I think this is typical of lots of emacs problems. You have all these modes that interact w/ each other, w/o strong models of the interactions, so you end up with a bunch of little bugs.
(And because there's combinatorially many different ways you can combine modes, I don't think testing is ever really going to help you here. You just need formal structures and coding styles that guarantee or lend themselves more to correctness.)
> There's a lot of cargo culted config fragments like the ones you'll find in the article.
Besides changing the gc-cons-threshold (which is indeed a questionable fragment to blindly copy/paste), there were no other specific config fragments, only examples. People blindly paste config fragments for everything (e.g. set nocompatible in Vim).
> Why does every user have to constantly reinvent the same config to work around all of the issues in Emacs?
What is there to reinvent? It should take almost no extra configuration to properly defer package loading. Package loading is also not automatic. If you are explicitly loading every package you've installed during init, that's your fault. You shouldn't blindly copy things into your config. If you really don't want to handle configuration yourself, starter kits exist for a reason.
> In the article the author talks about how he's gone bankrupt three times with his config, maybe that's a sign that the problem isn't the author. If Emacs is that fragile, then its extensibility is really a moot point.
It is mostly a user problem. Emacs lets you do anything. It gives you enough rope to hang yourself with. I could write an unreadable and broken mess of a program in any programming language, but that doesn't say anything about whether a language is fundamentally flawed.
It's a double-edged sword. There's no such thing as bug-free software. And the more functionality, the more room for misconfiguration and bugs. Vim/neovim come nowhere close in terms of functionality. The correct word isn't "fragile." A better word would be "malleable." It's the editable editor. Emacs' malleability is at least part of the reason why it's able to stay relevant despite the fact that it has very few developers compared to Vim and other editors. Because Emacs is this way, it's potentially easier to do something dumb or install a bunch of useless, slow packages, but because Emacs is this way we also have Magit, org-mode, and a bunch of other killer packages that Vim has much worse versions of (or no equivalent of at all).
This means that if you only need a simple setup, Emacs is not that appealing, but my experience has been that I have no idea what I want until I know what is actually possible (e.g. I didn't grok magit until I read the entire manual and saw how greatly it would simplify my workflow).
> Fast things feel nicer.
Daemon mode is fast. Mine takes less than a second to open a new frame with a fully-loaded Spacemacs setup, and while my startup time is 20-30 seconds, it runs in the background upon login, and I don't care about that gap.
> When customizing Emacs, you sometimes get into weird states that can be hard to recover from without restarting. For example, if you add a slow lambda function to your post-command-hook, it’s tough to remove it.
> Restarting Emacs helps verify that customization will persist between sessions.
These only matter if your primary use-case for Emacs is...customizing Emacs. Which is a thing that some people do, I guess, but I'm more interested in using Emacs as a tool. Sure, I'm going to spend dozens of hours tweaking it, but I'll spend thousands of hours using it - and rewriting my init.el (especially from its current Spacemacs state) is also going to take dozens of hours, which will definitely wipe out any gains that I would have gotten from having a slightly faster iteration time during the elisp development loop.
That said, I still think that the information presented in the article is useful! It's always good to know why software is slow and how to make it faster, and some people just don't want to use daemon mode.
I'm not sure all the extra effort needed really does the trick - I don't think mine would start up 'fast enough' to beat running a daemon even if I deferred loading most of my config (which can lead to config issues at times).
That being said, I had a look at their dotfiles, and it seems really well organized. If such an effort makes you write a great config, it may be worth it regardless of whether you end up using the daemon or not!
And these "advanced techniques" aren't really all that hard - 90% is just using use-package, which has a number of other advantages. It can be set up to install uninstalled packages, so you can do a self-installing init.el. In my case mine will download my other config files, install use-package and then install all packages I use.
It also leads to better organization of your config, because you keep the packages self-contained instead of splattering random settings all over init.el.
This way I get fast startup and the download problem you mention doesn't happen.
If nix isn't your thing, I used to do the same with straight.el by checking in .emacs.d/straight and disabling install by setting repos to nil.
project(ile)-switch-to-buffer is your friend.
Emacs (much like VSCode or Idea) is a tool that I use for everything, all day. I don't see a point in shutting it down, except for upgrading the version. Its startup time of, say, 15 or even 30 seconds (if you start many language servers) is only paid once, when I log in.
So I run it in server mode, and emacsclient start instantly, as a GUI window, or in a terminal, with all the amenities of an IDE already in place.
Do you really have a "different use case", or are you just averse to the idea of running it in the background as a daemon? Do you really need to save the 70 MB (without addons) to 200 MB (with addons) that it takes sitting there?
Unlike bloated Electron apps, Emacs doesn't actually use CPU while you're not using it - it just sits there, takes up a little memory (and even on my old 4 GB RAM ThinkPad it's not enough to actually inconvenience me), and awaits you opening a new frame with emacsclient.
But whether or not it solves my problem yes I think the use case is different? The person I responded to uses it so consistently they leave it running at all times. I can go days or weeks between single uses, depending on the projects I'm working on. That's a very different use to me.
Although, now, I'm sure that many of us are curious - why would you only use Emacs this infrequently, but bother to use it at all?
Plus I work remote and for better or worse vs code has become conventional for remote pairing. So I have a minimal emacs config for when I don't need any particular language integration, and the rest of the time vs code is good enough for the daily work I actually do, which is rarely writing out features from scratch anymore.
Jesus. I use Gvim, which starts in less than a second. When I'm done editing, I close it. The only app I generally leave open, is browser. How people can justify such awful performance is beyond me, and why I will never, ever use Emacs.
The question is actually about IDE features which take longer time to initialize, mostly in the non-Emacs code such as language servers.
Emacs with only built-in plugins loads from scratch instantly on my machine. But if I only need to do a simple quick edit, I can equally invoke vi :) (Which I regularly do, to keep my basic vi skills current.)
Now with vterm I run Emacs all the time, my buffers stay mostly open. Anyway, the best manner to use the tool is probably dictated by the tool, but I will tell you I am not bothered at all for my Doom Emacs 2 second starting time every few hours / days.
It's like complaining that it takes 30s to boot your PC when Gvim opens up instantly. It's a silly comparison.
I would also point out that early-init exists these days and is the best place for some startup logic (especially bits that prevent UI ever being shown so you don’t get weird flashes of stuff during startup).
- Doom emacs does most (all) of these with a bunch of cleverer techniques than mine. I’d recommend doom over rolling your own emacs.
- I use IntelliJ just about exclusively.
- I now do the minimal amount of customization I can get away with.
and aliases
ec='emacsclient -nc'
e='emacsclient -c'
emacsclient -nw
I stopped autoloading things, as i found it less bothersome to wait 30 seconds for Emacs to load than to get surprised by an autoload delay while working.
Use emacsclient if you want to use it as $EDITOR.
Because this is an old post, references like Spacemacs are now quite dated. Folks prefer Doom Emacs nowadays (though I don't use either of them).
Opening all buffers you've ever edited sounds nuts to me. What if one of the files you edited was a 10 gb log file or whatever?
(defun MY-consult-find-file (arg)
"If no prefix: find-file, otherwise consult-recentf."
(interactive "p")
(let* ((filename
(if (eq arg 1)
(call-interactively 'find-file)
(call-interactively 'consult-recent-file)))
)))
(global-set-key (kbd "C-x C-f") 'MY-consult-find-file)
(On the other hand, emacs is never quite problem free, so maybe this is introducing problems and I just haven't noticed.)
I have tons of customizations and lots of packages, but be as religious as reasonable about lazy loading.
$ time emacs --eval '(save-buffers-kill-emacs)'
real 0.21s
About half of it is unavoidable.
$ time emacs -q --eval '(save-buffers-kill-emacs)'
real 0.11s
(With default GC settings, startup is about 300ms.)
early-init.el:
(setq gc-cons-threshold most-positive-fixnum)
(add-hook 'emacs-startup-hook (lambda ()
(setq gc-cons-threshold (* 2 1024 1024))))A sibling poster recommended GCing on deactivate, which is an interesting idea!
But again, I truly haven't been able to notice any ill effects of not GCing. I tend to probably leave emacs open for maybe a few hours at a time, so I guess in that sense I'm manually GCing. :-)
Have you tried running 'M-x memory-report' before quitting to see how much memory you ended up using?
Anyway, here's the poor man's version:
cons-cells-consed is 38315202
There's probably some roughly accurate map from that to number of bytes of memory allocated, I will take a totally random guess it's 10, which would give 383MB of memory used.
I'm at half a day of usage, so figure 800MB by end of day. 16GB machine, so you can see why my "garbage collect once a day before bed" strategy is working out ;-).
(Plus also I wonder how much of the uncollected garbage just gets paged out by the OS and never touched again.)
(add-function :after
after-focus-change-function
(lambda () (unless (frame-focus-state)
(garbage-collect))))Just thought I'd add a side-note;
For anyone wanting to try emacs really give "Doom Emacs" a try ! It's brilliant and comes with vim-bindings (a.k.a evil-mode) !
Doom Emacs has spend a lot of time on making emacs(doom) fast !
(emacs-init-time)
"4.7 seconds"
(emacs-uptime)
"99 days, 3 hours, 14 minutes, 25 seconds"
On this point I agree, but another (better, in my opinion) option is to hack in one instance and spawn a second to check the config changes you're making. You can kill and recreate the spawned one with less interruption to your work.
I used my first Unix machine in 1991 and started college in 1993. I knew vi but the professor used emacs and showed us how to run make from within emacs. But wow was it slow! I asked an older student and he said EMACS was just like GNU and was a recursive acronym. Emacs Makes A Computer Slow and then told us Eight Megs And Constantly Swapping. The workstations had 16MB RAM at the time. One big problem was that vi was installed locally on every machine while Emacs was on a remote file server and there were probably 50 students typing "emacs" at the same time at the start of class all accessing the same server.
I just used vi and a bunch of terminals although later I used XEmacs in "viper" mode (vi keybindings) because it had color syntax highlighting and the split window support was cool. Vim got those features a long time ago so I just keep using vi/vim
2~3 seconds startup time with hundreds of packages.
The thing about helix is… it starts instantly and let’s say 90% of what I care about