Ask HN: How do I make a shell?
Does anyone have any links to tutorials or whatever, on how to go about creating a shell from scratch? I'd like to have a hand at creating something like Bash.
Next, recognise that a shell is a language, so you're basically building a commandline programming language interpreter (like python and ruby offer). The difference compared to a general-purpose language is that a shell language traditionally has super-lightweight syntax for operations related to processes and files.
The first step of building a language interpreter is being able to lex (tokenize) and parse a stream of characters.
Except it works with several OS interfaces touched by very little other programs (terminal mode, terminal's controlling process, process groups, job control), so one has to learn those somehow.
> Next, recognise that a shell is a language, so you're basically building a commandline programming language interpreter
No, not really. You can have something that just splits line on spaces and executes that as a program and arguments. Calling that a "language" is a quite far stretch.
What misled you is that most (all?) of the useful shells are programming languages at the same time as command editors and code executors (in other words, REPL). You just don't see what I described above, because it's awful to work with.
I originally put "user-mode executable" but edited it to "normal executable" to be more understandable.
I don't think it would be a stretch to call space-separated opaque values a language. A very simple one, mind.
My answer was directed at someone dipping their toe in who explicitly mentionined that they wanted to make a bash-esque (i.e. non-trivial language) shell.
Doing this in a very high level language like Ruby is trivial; it does all the work for you.
Doing this in C, as Bash does, requires learning some systems programming. Under Unix/Linux that means using fork() to create a child process and then having that child call exec() to run some specified command.
If not run "in the background", then the parent process needs to call some variant of wait() to idle until the child is done.
If it is desired to have input and/or output different than the parent, say for redirection of stdin/stdout/stderr, then one has to fiddle with variants of dup() to arrange this.
In some sense, all of the features mentioned in the other comments are optional, and the number of features in a sophisticated shell like Bash is mindboggling, so it would be a good idea to figure out a "minimum viable product" so that you can get your minimal desired feature set working before the end of time.
- get input from the user
- pipe it to a subprocess [1] -- ex. subprocess.call("somecmd", shell=True)
- show output to the user
- run this all inside a while true loop
Is it "cheating" to do it this way? Should the shell be responsible for parsing a language, generating an underlying representation like an abstract syntax tree, etc?
I'm speaking from the Python perspective but of course this idea is about the same in any very high level language.
[1]: https://docs.python.org/2/library/subprocess.html#using-the-...
> Is it "cheating" to do it this way? Should the shell be responsible for ...
Depends on your goals. There is no absolute answer.
If you just want to get things done fast, use Ruby. If you want to understand what Bash does, and learn more about systems programming, then do it the long way.
- can search $PATH for things to run (traversing $PATH),
- ?must check 'executable' bit?
- ?must discriminate scripts from binaries?
- must exec things to be run
- must intercept stdin, stdout, and stderr of things being run
- pass arguments to programs
- expand wild card characters (for UNIX shells. A in some sense more reasonable alternative is to put wildcard expansion in a separate library and let executables do wildcard expansion. That would prevent problems with expanded command lines becoming too long)
- allow for back tick substitutions
- evaluate environment variables when interpreting arguments (again, this _could_ be something left to programs to do. For this, I don't see a benefit, though)
- allow multiple commands on a line
- allow redirection from and to files
- build pipelines and execute them
- has some builtins (mostly optional, to speed up things)
- reading and interpreting .shellrc
- control structures (if, for, etc.)
You can implement them in whatever order you want.
I would probably start like this:
- a program that starts a single executable.
- starting an executable and intercepting its output.
- also sending it input
- accept the name of the command to run on the command line, search for it, and execute it.
- allow sending arguments to the program to run
- loop infinitely, running the executables whose names the user types in (this is where your program becomes shell-like)
You will likely encounter many things I let out above (do you need to interact with the terminal? How to handle Control-C, for example to nicely kill all commands in a pipeline) or do not even know about, but I would tackle them when I met them.
[1] http://brennan.io/2015/01/16/write-a-shell-in-c/
[2] http://www.ibm.com/developerworks/aix/library/au-pythocli/
[1] http://www.blackbytes.info/2016/07/writing-a-shell-in-ruby/