Felix Ple┼čoianu, 2009-06-07 (DRAFT)

Make Your Own Programming Language
Part 5

This is the last in a 5-part tutorial on how to implement a programming language. It is intended for people with some programming experience, who want to know how their compiler, interpreter or virtual machine works. Hint: it's not magic.

In Part 4 you've learned how to round up Scratch with lists and control structures. Our little language is more or less complete at this point, so we'll spend this installment drawing conclusion and making plans for the future.

Possible refinements

In hindsight, some parts of Scratch could be better (re)designed. Strings, for example. Having to type a space after the opening quote is just odd. This can be solved by pushing string handling into the lexer (where it belongs, in all honesty). But then the lexer has to take over word classification as well. Which is all for the best: the interpreter proper has better things to do. Just keep in mind that any special case muddles the code and may also slow it down.

That said, checking stack access should be the interpreter's job. It would simplify primitive definitions considerably.

Also, a practical implementation would focus on primitives that allow user programs to read ahead, manipulate the dictionary and control the compilation process. Then many of the more complicated features could be implemented in Scratch. Then again, you may not want to give the language users so much power, especially if you're going to use it as an embedded scripting language. Speaking of which...

Practical implementations

I did not have any practical uses in mind initially. Especially since, as a stack-based language, Scratch is so different from most others that an unsuspecting programmer might simply reject it. But there are many classes of applications that could use an embedded scripting language yet don't normally get one. Such as anything written in ActionScript or PHP. Being so compact, Scratch can be easily adapted to the specifics of the host language and of the intended task. Not to mention its inherent simplicity should ensure acceptable performance even on top of another interpreted language.

As of 2010-05-13 there is a third-party Ruby implementation. It encouraged me to display my own vision of what a real-world version might look like.

Further reading

Like with all programming, there are two parts to making your own programming language: design and implementation. For the latter, I can think of no better resource than Let's Build a Compiler, by Jack Crenshaw. Although it was never finished, this sprawling series will teach you a lot of things I decided not to cover here. At the other end of the verbosity spectrum, Peter Norvig's (How to Write a (Lisp) Interpreter (in Python)) does just what it says, and in only 90 lines of code! It's a less gentle introduction that my tutorial, but take a look anyway. (It also implements a less complete language; check out part 2 for the rest of it.) Ulithp is an even more compact implementation in Ruby.

Design is a more delicate matter. Just as you can't be a writer until you've read a LOT of books, you'll find it difficult to design programming languages until you've learned several existing ones. Then you can look at something like Syntax Across Languages or Comparison of programming languages (basic instructions) to see how all the languages you haven't learned implement various features.

Getting acquainted to a compiler compiler is also a good idea.

Last but not least, do your own research. There are various other resources out there, books, articles, communities, with which I'm not familiar enough to make recommendations. You should find your own path anyway. As for me... see you in the next tutorial.

Creative Commons License
Make Your Own Programming Language by Felix Pleşoianu is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License.