A game of tokens: solution - Part 2

Here we are, with my solution to the second part of "A Game of Tokens", aka how to write a simple interpreter in Python and survive to tell the tale. The second part of the challenge can be found here. Again, it is never repeated too many times, this is my personal solution. Your solution can be different, and if all the test pass it is correct!

You can find the code for this part in this repository. The branch called part2 contains all the commits explained in this post, and every commit contains both the test(s) and the code that makes the test(s) pass.

Level 8 - Multiplication and division


The tests we added for the lexer already pass. This is not surprising, as the lexer is designed to return everything it doesn't know as a LITERAL (smallcalc/calc_lexer.py:119). As we already ... more

A game of tokens: write an interpreter in Python with TDD - Part 2

My solution to the first part can be found here, and a discussion about the code has been published here Remember that this is only one possible solution, and that giving you the tests I already biased the whole project towards my implementation of choice, so feel free to experiment on your own!

In this instalment I'll give you new tests that will guide you through the implementation of a decent calculator, with multiplication, division, parenthesis, and unary operators. I will reference the structure I used in my solution, but you mileage may vary, so feel free to ignore the comments or the suggested solutions in case your code is different.

Level 8 - Multiplication and division

"They're coming outta the walls. They're coming outta the goddamn ... more

Refactoring with tests in Python: a practical example

This post contains a step-by-step example of a refactoring session guided by tests. When dealing with untested or legacy code refactoring is dangerous and tests can help us do it the right way, minimizing the amount of bugs we introduce, and possibly completely avoiding them.

Refactoring is not easy. It requires a double effort to understand code that others wrote, or that we wrote in the past, and moving around parts of it, simplifying it, in one word improving it, is by no means something for the faint-hearted. Like programming, refactoring has its rules and best practices, but it can be described as a mixture of technique, intuition, experience, risk.

Programming, after all, is craftsmanship.

The starting point

The simple use case I will use for this post is that of a service API that we can access, and that produces data in JSON format, namely a list of elements like the one shown here

... more

A game of tokens: solution - Part 1

This is my solution of the challenge posted here. As I stressed in that post, this is just one possible solution, and not even necessarily the best one. I provide it to show how I managed to solve the tests and how I worked in a TDD way.

Speaking of TDD I realised that I hadn't followed it very strictly, as sometimes I wrote more code than needed, usually forecasting future changes. I do not believe in a inflexible and uncompromising application of rules, so I do not consider this a big issue, as long as the result is a working code that is not blatantly overengineered.

You can find the code for this part in this repository. The branch called part1 contains all the commits explained in this post, and every commit contains both the test(s) and the code that makes the test(s) pass.

Level 1 - End of ... more

A game of tokens: write an interpreter in Python with TDD - Part 1


Writing an interpreter or a compiler is usually considered one of the greatest goals that a programmer can achieve, and with good reason. I do not believe the importance of going through this experience is primarily due to its difficulty. After all, writing an efficient compiler is difficult, but the same is true for a good web framework, or a feature-rich editor.

Being able to write an interpreter is a significant skill mainly because of its recursive (or self-referring) nature. Think about it: you use a language to write a new language. And this new language, if it becomes sufficiently rich, can eventually be used to create its own compiler.

A language can be used to write the program that executes that same language.

Didn't this last sentence fire you with enthusiasm? It makes me eager to start!

Compilers have been the subject of academic research since the 50s, with the works of ... more

Clean architectures in Python: a step-by-step example

One year ago I was introduced by my friend Roberto Ciatti to the concept of Clean Architecture, as it is called by Robert Martin. The well-known Uncle Bob talks a lot about this concept at conferences and wrote some very interesting posts about it. What he calls "Clean Architecture" is a way of structuring a software system, a set of consideration (more than strict rules) about the different layers and the role of the actors in it.

As he clearly states in a post aptly titled The Clean Architecture, the idea behind this design is not new, being built on a set of concepts that have been pushed by many software engineers over the last 3 decades. One of the first implementations may be found in the Boundary-Control-Entity model proposed by Ivar Jacobson in his masterpiece "Object-Oriented Software Engineering: A Use Case Driven Approach" published in 1992, but Martin lists ... more

Python Mocks: a gentle introduction - Part 2

In the first post I introduced you to Python mocks, objects that can imitate other objects and work as placeholders, replacing external systems during unit testing. I described the basic behaviour of mock objects, the return_value and side_effect attributes, and the assert_called_with() method.

In this post I will briefly review the remaining assert_* methods and some interesting attributes that allow to check the calls received by the mock object. Then I will introduce and exemplify patching, which is a very important topic in testing.

Other assertions and attributes

The official documentation of the mock library lists many other assertion, namely assert_called_once_with(), assert_any_call(), assert_has_calls(), assert_not_called(). If you ... more

Punch - Update your version while having a drink

So you completed your wonderful new project, all your test are successful (you test code, don't you?) and you just want to ship the new version and call it a day. Well, you just have to go and change the version number in your install script and save. Oh, right, you also have to open a feature branch, so that you may record the version update in your Git history. Well, easily done. Damn! You forgot to change the version number in the README.md file...

Managing the version number of a project is not easy. Not only you need to think about the versioning scheme and what part of the version to increase (see this post for some tips on this matter), but you also need to remember in which files you put the actual version number, and, depending on your workflow, to correctly manage the version control system commits.

Punch is a small tool that aims to simplify the latter parts, that is ... more

Abstract Base Classes in Python

With the introduction of Abstract Base Classes, Python once again shows its nature of a very innovative and flexible language. It is interesting to see how such a remarkable feature has been introduced into the language by a pure Python module. This demonstrates that Python is built in a way that is very open to changes, thanks to its foundations in pure polymorphism based on delegation.

Many Python programmers overlooked Abstract Base Classes and the classes in the collections module, which are one of the simplest and useful applications of the concept. Sure enough, this is not a feature that you will use every day or that will change the way you are programming in Python. But neither is it something you shall discard before understanding what it brings into the language, and what sort of problems it can solve for you.

Level 1


Python is a dynamically-typed object-oriented language strongly based on delegation, so its approach to problems is intrinsically ... more

Python Mocks: a gentle introduction - Part 1

As already stressed in the two introductory posts on TDD (you can find them here) testing requires to write some code that uses the functions and objects you are going to develop. This means that you need to isolate a given (external) function that is part of your public API and demonstrate that it works with standard inputs and in edge cases.

For example, if you are going to develop an object that stores percentages (such as for example poll results), you should test the following conditions: the class can store a standard percentage such as 42%, the class shall give an error if you try to store a negative percentage, the class shall give an error if you store a percentage greater than 100%.

Tests shall be idempotent and isolated. Idempotent in mathematics and computer science identifies a process that can be run multiple times without changing the status of the system. Isolated means that a test shall not change its behaviour depending on ... more