This article contains affiliate links. See my affiliate disclosure for more information.
Everyone wants to write clean code. There are whole books about it!
But you don't need to read a book to write cleaner code right now. There's one "trick" that every coder can learn to make their code less confusing.
The key is:
One line, one task.
But don't go crazy with it.
Here's the main idea: Short lines of code require less brainpower to read than long ones. Code that's easy to read is easier to reason about. Programs with shorter lines are, in theory, easier to maintain.
But compact code can be cryptic. (Ever seen APL?) And just because you can split a line doesn't mean you should.
In some languages, you can assign two values to two variables on one line:
x, y = 2, 7
You could put both assignments on their own line:
x = 2
y = 7
But, c'mon. Do you really need to? How can you tell if a line should be split up?
Felienne Hermans opens her book The Programmer's Brain with an undeniable truth: "Confusion is a part of programming."
Hermans' book (which I highly recommend) explains how your brain's three memory functions work together to understand code:
STM and WM are small. Both can only store about 4 to 6 things at a time! Overload them and you've got a recipe for confusion.
That gives us a rule for deciding if a line of code is too complex:
A line of code containing 6+ pieces of information should be simplified.
I call it the "rule of six."
Here's an example in Python:
map(lambda x: x.split('=')[1], s.split('?')[1].split('&')[-3:])
Is that hard for you to read? Me too. There's a good reason why.
You have to know what map
, lambda
, and .split()
are. The variables x
and s
, the strings '='
, '?'
, and '&'
, the index [1]
, and the slice [-3:]
all take up space in STM and WM. In total: ten things! Your brain can't keep up.
Or maybe yours can.
If so, you've got some good experience under your belt.
Your brain "chunks" syntax like s.split('?')[1]
into "the part of the string to the right of the question mark." And you can reconstruct the code using information stored in your LTM. But you still only process a few chunks at a time.
So… we can identify when a line of code is too complex. Now what?
One email, every Saturday, with one actionable tip.
Always less than 5 minutes of your time.
Break it into smaller pieces, that is!
There are two strategies I use to break up code. I call them SIMPLE and MORF.
The SIMPLE strategy adds lines of code to decrease cognitive load.
Let's apply SIMPLE to that nasty one-liner we saw earlier. Remove the second argument from map()
and put it on its own line:
query_params = s.split('?')[1].split('&')[-3:]
map(lambda x: x.split('=')[1], query_params)
It still might be hard to read. There are seven things to keep track of in the first line:
query_params
s
.split()
'?'
[1]
'&'
[-3:]
But each line has fewer things to track than before. Your brain can process them more easily.
Apply SIMPLE again and move s.split('?')[1]
to a new line:
url_query_string = s.split('?')[1]
query_params = url_query_string.split('&')[-3:]
map(lambda x: x.split('=')[1], query_params)
Compare that to the original one-liner. Which one is easier to process?
The MORF strategy takes a different approach and groups code into functions.
Here's what MORF looks like applied to our one-liner:
def query_params(url):
return url.split('?')[1].split('&')[-3:]
map(lambda x: x.split('=')[1], query_params(s))
You can even combine MORF and SIMPLE:
def query_params(url):
query_string = url.split('?')[1]
return query_string.split('&')[-3:]
map(lambda x: x.split('=')[1], query_params(s))
You don't have to understand the code to feel the effect. Each line is easier for your brain to process.
There's a bonus benefit, too!
Once you know that your WM and STM aren't overloaded, you know that any confusion left over is due to missing information in your LTM.
In other words, SIMPLE and MORF don't just help you write cleaner code. They help you identify knowledge gaps that you can improve with practice!
Want to know more about how your brain works while you're coding?
Check out The Programmer's Brain by Felienne Hermans.
Get instant access through Manning, or order it from Amazon.
Look at the code we ended up with using SIMPLE:
url_query_string = s.split('?')[1]
query_params = url_query_string.split('&')[-3:]
map(lambda x: x.split('=')[1], query_params)
One line still has over six "ideas" in it and should, according to the rule of six, be split up:
You can do a lot with very little.
Modern, high-level languages abstract away lots of details. You don’t need to know the difference between selection sort, bubble sort, merge sort, and quicksort, for instance, to use Python’s sorted()
function. It just works.
Do you really need to learn how to use data structures and algorithms?
Well… I've got news for you:
Ever used a string? That’s a data structure. Written a for
loop? That’s an algorithm.
Algorithms and data structures are the building blocks of computer programs — even “Hello, World!” Over time, programmers recognize useful patterns they can apply to a broad range of problems. Language designers bake the most commonly used patterns into easy-to-use interfaces.
You don’t have to know how they’re implemented to reap the benefits.
New coders, especially, need to avoid falling off the deep end.
Let’s be realistic.
You can solve at least 95% of coding problems without any deep knowledge of algorithms and data structures. Beginners need to focus on learning syntax and getting proficient at translating ideas expressed in a human language into code. You don't want to spend all of your energy on something with little immediate impact.
But here’s the rub:
You could end up with your hands tied.
You may write code that works fine in small settings but screeches to a halt on larger workloads. You have a scalability problem. Or you may become so dependent on third-party packages that you can’t solve problems that hit edge cases. You have a specialization problem.
Not everyone that drives a car needs to be a mechanic, though. So how do you know it’s time to take the plunge?
One email, every Saturday, with one actionable tip.
Always less than 5 minutes of your time.
Will your code need to:
In each of these scenarios, data structures and algorithms are vital. Not because you’ll need to write everything from scratch — although, you might — but because you’ll need tools to evaluate and compare solutions. For some projects, it can be the difference between success and failure.
But there's another reason to learn algorithms and data structures.
One that every coder should know:
One of my favorite quotes about art is from the English painter David Hockney:
Art has to move you, design does not, unless it’s a good design for a bus.
A good algorithm is a work of art. The ingenuity of its inner workings, the cleverness with which it skirts constraints, can elevate an algorithm from the mundane to the downright magical.
Yes, studying algorithms and data structures may be necessary for your career.
Then again, it might not.
But not everything worth learning has to be immediately useful.
Here are some of my favorite books on algorithms and data structures:
For the curious beginner:
For folks who want to dig deeper:
For folks who want all the details:
Are you fine following along with coding tutorials, but feel stuck starting anything alone? You, my friend, are stuck in tutorial hell.
I thought I'd share some ideas I've used myself to boost creativity and build independence.
Most of the suggestions I've heard for getting out of this situation are vague and unhelpful. "Just build something," they say! 😒
They're not wrong, necessarily, but it's terrible advice. It isn't actionable.
Here are three specific ways to escape tutorial hell 👇
What's something you do every day or several times each month? Can you automate it? Here are some things I've automated in the past:
Why it's a good idea: You'll work with APIs, webhooks, email, and task runners. More importantly: you'll stay motivated because it provides real value.
How to make the most of it: Minimal features! Don't go overboard. Keep things simple and focused. You can always add new features later.
One email, every Saturday, with one actionable tip.
Always less than 5 minutes of your time.
Make your own versions of Unix commands like mv
, cp
, and rm
. Just don't accidentally erase your hard drive re-creating rm
😳
Why it's a good idea: You'll practice accepting input to a program, handling errors, and working with the file system.
How to make the most of it: Keep it simple! Shell commands can be surprisingly complex, but it's usually straightforward to implement the basics.
I know this seems counterintuitive… but bear with me! If you're unable to start a coding project without using a tutorial, why not try something other than coding?
Why it's a good idea: Writing a tutorial is different from following a tutorial. You'll be forced to think about things from a different angle.
How to make the most of it: Keep it short! Pick a single function, package, or concept you've recently learned about and write your own explanation of it. Ask a friend or colleague to review it for you.
Links marked with ^{*} are affiliate links. Refer to my affiliate disclosure for more information.
I understand entirely if "love" isn't something that math evokes in you. It didn't for me, either, until I was in my early twenties.
As a high school student, I felt relieved when I completed the required math credits and could skip calculus my senior year. Had anyone told me I'd eventually major in mathematics — even pursue a doctorate in it! — I'd have laughed in their face.
I remember the moment things changed. It was 2005, and I was living in Los Angeles. I'd grown my hair out and was playing keyboards in a rock band. My roommate — a boisterous Londoner and the band's drummer — had read Bill Bryson's A Short History of Nearly Everything^{*} and encouraged me to read it. I didn't.
I mean, I was going to. I went to Barnes & Noble and headed straight for the physics section. I saw the Bill Bryson book on the shelf, but something else caught my eye: The Elegant Universe^{*} by Brian Greene. I probably read a hundred pages of Greene's book in the store. I forgot all about Bryson and walked out of Barnes & Noble that day with a new hunger to understand something called "string theory."
As it turns out, physics is a pretty effective gateway drug to the world of mathematics. Within two years of reading The Elegant Universe, I'd left the world of music and entered university ready to study physics. But thanks to a superb calculus teacher, I caught the math bug instead. I graduated four years later with a degree in mathematics. I never even took a physics course.
At the same time that I became interested in string theory, the U.S. government was funding a group of mathematicians and theoretical physicists to connect ideas from string theory to a deep problem in mathematics called the Langlands Program. And one of the principal researchers — a mathematician named Edward Frenkel — had, like me, begun his mathematical life with an interest in physics.
As I read Frenkel's 2013 book Love & Math^{*}, I couldn't help but smile as I thought about how, at the same time that a 20-year-old kid in Los Angeles was just starting his mathematical life enamored by string theory, there was a mathematician at Princeton exploring related ideas and experiencing the culmination of a life of study brought on by the same impulse I'd felt: a desire to understand the universe.
I've never met Edward Frenkel and probably never will (although I may have once been in the same room with him — we both attended the 2012 Joint Mathematics Meetings). But I can't help but feel a connection to his story. Reading Love & Math kicked up a whirlwind of emotions and rekindled an interest in an old flame.
Part autobiography, part math explainer, and part romance novel, Love & Math^{*} is, at its core, an invitation for everyone to experience what mathematicians have largely kept to themselves.
Frenkel's goal is to "unlock the power and beauty of mathematics, and enable you to enter this magical world… even if you are the sort of person who has never used the words 'math' and 'love' in the same sentence." He aims to level the playing field:
It's too precious to be given away to the "initiated few." It belongs to all of us.
Frenkel alludes to elitism^{[1]}, but his desire to democratize mathematics may be rooted in something far more insidious. See, Frenkel grew up in Soviet Russia during the 1970s and 1980s. And although his mathematical talents were obvious, state-sponsored anti-Semitism (Frenkel's father is Jewish) kept him from attending the best Russian universities.
In one incredible story, Frenkel recounts how during his entrance exam to the Mekh-Mat, the "flagship mathematics program of the USSR," one of his interviewers asked him to define a circle. Frenkel answered correctly, "A circle is the set of points on the plane equidistant from a given point." The man told him he was wrong, that a circle is "the set of all points on the plane equidistant from a given point." Frenkel notes that this "sounded like an excessive parsing of words." But, really, it was just racism.
The autobiographical aspects of Love & Math^{*} are superbly well-written. They tell the story of a young man from Russia who, against all odds, persevered and fulfilled his dream of becoming a professional mathematician. In doing so, the reader glimpses far more than Frenkel's life story. You get to experience the drama — the humanity — that's inherent in doing mathematics.
In one chapter, Frenkel dramatizes a discussion with another Russian mathematician, Vladimir Drinfeld, that shaped his mathematical career. I mean "dramatizes" quite literally: Frenkel presents the discussion as a screenplay. It's one of the best portrayals I've encountered of the social component of mathematical research. It reminded me of the talks I'd have with my advisor in her office while working on my thesis^{[2]}.
Frenkel even describes the emotional experience of mathematical research, beginning with the frustration often encountered when first tackling a challenging problem:
It didn't come easily to me. It never does. I tried many different methods. As each of them failed, I felt increasingly frustrated and anxious.
This was Frenkel's first "real" crack at solving a research problem, and, as so many people do, he experienced imposter syndrome:
I questioned whether I could be a mathematician.
It's significant to me that Frenkel chose to include the anguish and doubt he experienced while working on a math problem. So often, people envision the big players in math as consummate geniuses. But, I'd bet money that even the likes of Gauss and Euler had their faith shaken from time to time. They were mere mortals, after all^{[3]}.
Frenkel spent "endless hours" with his problem, toiling over nights and weekends. His struggle led to insomnia, giving him a taste of the "side effects" that unhealthy obsession can ignite.
After months of sleepless nights and self-doubt, the answer came "suddenly, as if in a stroke of black magic." Frenkel describes the feeling of finally solving the problem as a "high." I think most people can relate to this. It's not any different than the dopamine rush you get from completing a challenging video game. But then Frenkel describes what, in my mind, is the true allure of mathematics:
For the first time in my life, I had in my possession something that no one else in the world had. I was able to say something new about the universe.
Humans are explorers by nature. Curiosity has taken us to the ocean floor. It has landed us on the moon. However, few locations remain to explore on our planet. The sciences offer a means to investigate the physical universe in new ways. Still, not everyone can afford space telescopes, particle colliders, and electron microscopes.
In contrast, mathematics provides the opportunity to explore the universe with a single tool available free of charge to every living person: your mind.
Interwoven among the autobiographical sections of Love & Math^{*}, Frenkel tells another story: the story of the math behind the Langlands Program. Named after Canadian mathematician Robert Langlands, the Langlands Program connects disparate ideas in mathematics with a kind of "Rosetta Stone."
As Frenkel's father put it, the details of the Langlands Program are "quite heavy." Frenkel admits, "even among specialists, very few people know the nuts and bolts of all the elements." That doesn't stop Frenkel from giving some descriptions of what's going on. And although the math in Love & Math can sometimes leave your head swimming, it serves a purpose:
My point is not for you to learn [it] all. Rather, I want to indicate the logical connections between these objects and show the creative process of scientists studying them: what drives them, how they learn from each other, how the knowledge they acquire is used to advance our understanding of the key questions.
Frenkel expertly guides the reader through concepts typically reserved for upper-level undergraduate — and even graduate-level — mathematics courses. And he does so in a way that meets readers where they are, without assuming much background.
This ability to distill technical concepts into everyday language is apparently a skill Frenkel learned from one of his mentors, Israel Gelfand. In the book's preface, Frenkel shares how Gelfand described this technique:
People think they don't understand math, but it's all about how you explain it to them. If you ask a drunkard what number is larger, 2/3 or 3/5, he won't be able to tell you. But if you rephrase the question: what is better, 2 bottles of vodka for 3 people or 3 bottles of vodka for 5 people, he will tell you right away: 2 bottles for 3 people, of course.
This approach works, and Love & Math^{*} is the proof. In Chapter 2, The Essence of Symmetry, Frenkel describes the concept of a symmetry group by examining a round table. Most people know what symmetry is, but mathematicians use the word "group" differently than we do in everyday English. Everyone has seen a round table and can easily imagine interacting with it. So when Frenkel dives into things like the group of rotational symmetries, you have a concrete image to work with.
Many times, Frenkel presents concepts as a narrative. Rather than slap you across the face with definitions and theorems, Frenkel shares how he learned the concepts himself. He explains rotational symmetries, for example, in a dialog between himself and his teenage mentor^{[4]}, Evgeny Evgenievich Petrov, a professor at a local college and friend of Frenkel's parents.
Frenkel readily shares his own confusion during these conversations, which helps put the reader at ease. One can imagine expressing their confusion to Frenkel and him responding with a smile: "Hey, these things aren't always easy to understand, and it's okay to struggle. But it's worth the effort, and thinking of it this way helped me finally grasp it."
All this is not to say that you won't encounter any serious math. As Frenkel points out, "It is perfectly fine to skip those parts that look confusing or tedious at the first reading (this is what I often do myself). Coming back to those parts later, equipped with newly gained knowledge, you might find the materials easier to follow. But that is usually not necessary in order to be able to follow what comes next."
Even I, with the advantage of formal training in mathematics, had to mull over several parts of the text. And although this might seem scary, I think it adds richness to the book. You don't need to understand everything at first, or even at all, to get the most value from Love & Math^{*}: an appreciation of what mathematicians do and how they do it. But if you want to go deeper, it's there for you.
The book's climax deals with a curious property of mathematics: ideas developed without any ties to the physical world sometimes, and often mysteriously, materialize as descriptions of some natural process. For the Langlands Program, this connection comes from quantum mechanics and, in particular, supersymmetry.
From 2004 to 2013, Frenkel led a research group investigating connections between supersymmetry and the Langlands Program. Although supersymmetry has yet to be tested experimentally, it's a model for understanding the universe. Somewhat miraculously, elements of supersymmetry correspond nicely to elements in the Langlands Program. It's as if all of these abstractions that, so far, exist only in the collective minds of mathematicians yearn to break free of the bonds of solipsism and claim their material existence.
Whether or not supersymmetry accurately describes the universe, the connection to the Langlands Program is real. And it embodies Frenkel's point that pursuing mathematics can lead to a profound understanding of the world around us. Mathematicians are, in some ways, like astronauts who can journey across the cosmos without ever leaving Earth.
There's a quote I've seen attributed to the German mathematician David Hilbert who, upon learning that a student had dropped his class to study poetry, apparently retorted, "You know, for a mathematician, he did not have enough imagination. But he has become a poet, and now he is fine.^{[5]}"
I get what Hilbert is saying. Too often, the role of creativity in mathematics is understated, if even mentioned at all. I've met people who think that my interest in math means that I'm an excellent accountant^{[6]}. But, if the story is, in fact, true, then I can't help but wonder if Hilbert lacked the imagination required to reach certain students. Perhaps he needed a mentor, like Israel Gelfand, who saw communicating mathematics as an exercise in empathy.
Empathy. That's a word I keep returning to in my mind when I try to pinpoint what I enjoyed most about Love & Math^{*}. Frenkel writes with genuine empathy — a kindness that seems to be born from an appreciation for the love of his parents, mentors, and all of the people that helped make his dream of becoming a mathematician possible, even when an anti-Semitic government put up roadblocks every step of the way.
Love & Math's final chapter, titled Searching For the Formula of Love, is both a memoir of Frenkel's excursions into the arts (he co-wrote a screenplay, The Two Body Problem, and a short film, Rites of Love and Math) and a manifesto for Frenkel's vision of a future where widespread appreciation for mathematics is normal. I admire Frenkel's willingness to pursue artistic interests. I wouldn't be surprised if his experience with film and theater helped develop the impressive storytelling he displays in Love & Math.
In Rites of Love and Math, a mathematician discovers "the formula for love." But there's a dark side, and, not unlike the discovery of atomic energy, the mathematician finds that his formula can be used for evil. "He realizes that he has to hide the formula," Frenkel writes, "to protect it from falling into the wrong hands. And he decides to tattoo it on the body of the woman he loves."
The film is an allegory for the "deeply personal experience" of creating new mathematics:
It requires love and dedication, a struggle with the unknown and oneself, which elicits strong emotions. And the formulas you discover really do get under your skin, just like the tattooing in the film.
Rites of Math and Love appeals to a general audience. By "emphasizing the human and spiritual elements of mathematics," Frenkel hoped to "inspire viewer's curiosity." In many ways, Love & Math does the same thing, except it doesn't shy away from digging into technical details. It's not just a love letter to mathematics; it's an invitation for readers to get their hands dirty and, possibly, discover something to love about math for themselves.
And so, Love & Math^{*} serves two purposes: first, it's an account of the human, emotional experience of doing mathematics, and second, it's an introduction to the kinds of problems that unite people across disciplines and probe the secrets of the universe.
A mathematical formula does not explain love, but it can carry a charge of love.
"Among other things," Frenkel writes, "mathematics gives us a rationale and an additional capacity to love each other and the world around us."
You can purchase Love & Math from Amazon^{*}, Barnes & Noble, iTunes, and IndieBound.
Thanks to Jeremy Alm for reading the draft of this post and providing valuable feedback.
When you learn to program for the first time, you look for — or, perhaps, are assigned — projects that reinforce basic concepts. But how often do you, once you've attained more knowledge and experience, revisit those beginner projects from the perspective of an advanced programmer?
In this article, I want to do just that. I want to revisit a common beginner project — implementing the game "Rock Paper Scissors" in Python — with the knowledge I've gained from nearly eight years of Python programming experience.
Before diving into code, let's set the stage by outlining how "Rock Paper Scissors" is played. Two players each choose one of three items: rock, paper, or scissors. The players reveal their selection to each other simultaneously and the winner is determined by the following rules:
Growing up, my friends and I used "Rock Paper Scissors" to solve all sorts of problems. Who gets to play first in a one-player video game? Who gets the last can of soda? Who has to go pick up the mess we just made? Important stuff.
Let's lay out some requirements for the implementation. Rather than building a full-blown game, let's focus on writing a function called play()
that accepts two string arguments — the choice of "rock"
, "paper"
, or "scissors"
selected by each player — and returns a string indicating the winner (e.g., "paper wins"
) or if the game results in a tie (e.g., "tie"
).
Here are some examples of how play()
is called and what it returns:
>>> play("rock", "paper")
'paper wins'
>>> play("scissors", "paper")
'scissors wins'
>>> play("paper", "paper")
'tie'
If one or both of the two arguments are invalid, meaning they aren't one of "rock"
, "paper"
, or "scissors"
, then play()
should raise some kind of exception.
play()
should also be commutative. That is, play("rock", "paper")
should return the same thing as play("paper", "rock")
.
To set a baseline for comparison, consider how a beginner might implement the play()
function. If this beginner is anything like I was when I first learned to program, they'd probably start writing down a whole bunch of if
statements:
def play(player1_choice, player2_choice):
if player1_choice == "rock":
if player2_choice == "rock":
return "tie"
elif player2_choice == "paper":
return "paper wins"
elif player2_choice == "scissors":
return "rock wins"
else:
raise ValueError(f"Invalid choice: {player2_choice}")
elif player1_choice == "paper":
if player2_choice == "rock":
return "paper wins"
elif player2_choice == "paper":
return "tie"
elif player2_choice == "scissors":
return "rock wins"
else:
raise ValueError(f"Invalid choice: {player2_choice}")
elif player1_choice == "scissors":
if player2_choice == "rock":
return "rock wins"
elif player2_choice == "paper":
return "scissors wins"
elif player2_choice == "scissors":
return "tie"
else:
raise ValueError(f"Invalid choice: {player2_choice}")
else:
raise ValueError(f"Invalid choice: {player1_choice}")
Strictly speaking, there's nothing wrong with this code. It runs without error and meets all of the requirements. It's also similar to a number of high-ranking implementations for the Google search "rock paper scissors python."
Experienced programmers will quickly recognize a number of code smells, though. In particular, the code is repetitive and there are many possible execution paths.
One email, every Saturday, with one actionable tip.
Always less than 5 minutes of your time.
One way to implement "Rock Paper Scissors" from a more advanced perspective involves leveraging Python's dictionary type. A dictionary can map items to those that they beat according to the rules of the game.
Let's call this dictionary loses_to
(naming is hard, y'all):
loses_to = {
"rock": "scissors",
"paper": "rock",
"scissors": "paper",
}
loses_to
provides a simple API for determining which item loses to another:
>>> loses_to["rock"]
'scissors'
>>> loses_to["scissors"]
'paper'
A dictionary has a couple of benefits. You can use it to:
KeyError
With this in mind, the play()
function could be written as follows:
def play(player1_choice, player2_choice):
if player2_choice == loses_to[player1_choice]:
return f"{player1_choice} wins"
if player1_choice == loses_to[player2_choice]:
return f"{player2_choice} wins"
if player1_choice == player2_choice:
return "tie"
In this version, play()
takes advantage of the built-in KeyError
raised by the loses_to
dictionary when trying to access an invalid key. This effectively validates the players' choices. So if either player chooses an invalid item — something like "lizard"
or 1234
— play()
raises a KeyError
:
>>> play("lizard", "paper")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 2, in play
KeyError: 'lizard'
Although the KeyError
isn't as helpful as a ValueError
with a descriptive message, it still gets the job done.
The new play()
function is much simpler than the original one. Instead of handling a bunch of explicit cases, there are only three cases to check:
player2_choice
loses to player1_choice
player1_choice
loses to player2_choice
player1_choice
and player2_choice
are the sameThere's a fourth hidden case, however, that you almost have to squint to see. That case occurs when none of the other three cases are true, in which case play()
returns a None
value.
But... can this case ever really occur? Actually, no. It can't. According to the rules of the game, if player 1 doesn't lose to player 2 and player 2 doesn't lose to player 1, then both players must have chosen the same item.
In other words, we can remove the final if
block from play()
and just return "tie"
if neither of the other two if
blocks execute:
def play(player1_choice, player2_choice):
if player2_choice == loses_to[player1_choice]:
return f"{player1_choice} wins"
if player1_choice == loses_to[player2_choice]:
return f"{player2_choice} wins"
return "tie"
We've made a tradeoff. We've sacrificed clarity — I'd argue that there's a greater cognitive load required to understand how the above play()
function works compared to the "beginner" version — in order to shorten the function and avoid an unreachable state.
Was this trade-off worth it? I don't know. Does purity beat practicality?
The previous solution works great. It's readable and much shorter than the "beginner" solution. But it's not very flexible. That is, it can't handle variations of "Rock Paper Scissors" without rewriting some of the logic.
For instance, there's a variation called "Rock Paper Scissors Lizard Spock" with a more complex set of rules:
How can you adapt the code to handle this variation?
First, replace the string values in the loses_to
dictionary with Python sets. Each set contains all of the items that lose to the corresponding key. Here's what this version of loses_to
looks like using the original "Rock Paper Scissors" rules:
loses_to = {
"rock": {"scissors"},
"paper": {"rock"},
"scissors": {"paper"},
}
Why sets? Because we only care about what items lose to a given key. We don't care about the order of those items.
To adapt play()
to handle the new loses_to
dictionary, all you have to do is replace ==
with in
to use a membership check instead of an equality check:
def play(player1_choice, player2_choice):
# vv--- replace == with in
if player2_choice in loses_to[player1_choice]:
return f"{player1_choice} wins"
# vv--- replace == with in
if player1_choice in loses_to[player2_choice]:
return f"{player2_choice} wins"
return "tie"
Take a moment to run this code and verify that everything still works.
Now replace loses_to
with a dictionary implementing the rules for "Rock Paper Scissors Lizard Spock." Here's what that looks like:
loses_to = {
"rock": {"scissors", "lizard"},
"paper": {"rock", "spock"},
"scissors": {"paper", "lizard"},
"lizard": {"spock", "paper"},
"spock": {"scissors", "rock"},
}
The new play()
function works with these new rules flawlessly:
>>> play("rock", "paper")
'paper wins'
>>> play("spock", "lizard")
'lizard wins'
>>> play("spock", "spock")
'tie'
In my opinion, this is a great example of the power of picking the right data structure. By using sets to represent all of the items that lose to a key in the loses_to
dictionary and replacing ==
with in
, you've made a more general solution without having to add a single line of code.
One email, every Saturday, to challenge your skills and inspire your curiosity.
Always less than 5 minutes of your time.
Let's step back and take a slightly different approach. Instead of looking up items in a dictionary to determine the winner, we'll build a table of all possible inputs and their outcomes.
You still need something to represent the rules of the game, so let's start with the loses_to
dict from the previous solution:
loses_to = {
"rock": {"scissors"},
"paper": {"rock"},
"scissors": {"paper"},
}
Next, write a function build_results_table()
that takes a rules dictionary, like loses_to
, and returns a new dictionary that maps states to their results. For instance, here's what build_results_table()
should return when called with loses_to
as its argument:
>>> build_results_table(loses_to)
{
{"rock", "scissors"}: "rock wins",
{"paper", "rock"}: "paper wins",
{"scissors", "paper"}: "scissors wins",
{"rock", "rock"}: "tie",
{"paper", "paper"}: "tie",
{"scissors", "scissors"}: "tie",
}
If you think something looks off there, you're right. There are two things wrong with this dictionary:
{"rock", "rock"}
can't exist. Sets can't have repeated elements. In a real scenario, this set would look like {"rock"}
. You don't actually need to worry about this too much. I wrote those sets with two elements to make it clear what those states represent.{"rock", "paper"}
and {"paper", "rock"}
evaluate equal to each other and should therefore return the same result upon lookup.The way to get around this is to use Python's built-in frozenset
type. Like sets, frozensets
support membership checks, and they compare equal to another set
or frozenset
if and only if both sets have the same members. Unlike standard sets, however, frozenset
instances are immutable. As a result, they can be used as dictionary keys.
To implement build_results_table()
you could loop over each of the keys in the loses_to
dictionary and build a frozenset
instance for each of the strings values in the set corresponding to the key:
def build_results_table(rules):
results = {}
for key, values in rules.items():
for value in values:
state = frozenset((key, value))
result = f"{key} wins"
results[state] = result
return results
This gets you about halfway there:
>>> build_results_table(loses_to)
{frozenset({'rock', 'scissors'}): 'rock wins',
frozenset({'paper', 'rock'}): 'paper wins',
frozenset({'paper', 'scissors'}): 'scissors wins'}
The states that result in a tie aren't covered, though. To add those, you need to create frozenset
instances for each key in the rules
dictionary that map to the string "tie"
:
def build_results_table(rules):
results = {}
for key, values in rules.items():
# Add the tie states
results[frozenset((key,))] = "tie" # <-- New
# Add the winning states
for value in values:
state = frozenset((key, value))
result = f"{key} wins"
results[state] = result
return results
Now the value returned by build_results_table()
looks right:
>>> build_results_table(loses_to)
{frozenset({'rock'}): 'tie',
frozenset({'rock', 'scissors'}): 'rock wins',
frozenset({'paper'}): 'tie',
frozenset({'paper', 'rock'}): 'paper wins',
frozenset({'scissors'}): 'tie',
frozenset({'paper', 'scissors'}): 'scissors wins'}
Why go through all this trouble? After all, build_results_table()
looks more complicated than the play()
function from the previous solution.
You're not wrong, but I want to point out that this pattern can be quite useful. If there are a finite number of states that can exist in a program, you can sometimes see dramatic boosts in speed by precalculating the results for all of those states. This might be overkill for something as simple as "Rock Paper Scissors," but could make a huge difference in situations where there are hundreds of thousands or even millions of states.
One real-world scenario where this type of approach makes sense is the Q-learning algorithm used in reinforcement learning applications. In that algorithm, a table of states — the Q-table — is maintained that maps each state to a set of probabilities for some pre-determined actions. Once an agent is trained, it can choose an action based on the probabilities for an observed state and then act accordingly.
Often, a table like the one generated by build_results_table()
is computed and then stored in a file. When the program runs, the pre-computed table gets loaded into memory and then used by the application.
So, now that you have a function that can build a results table, assign the table for loses_to
to an outcomes
variable:
outcomes = build_results_table(loses_to)
Now you can write a play()
function that looks up the state in the outcomes
table based on the arguments passed to play and then returns the result:
def play(player1_choice, player2_choice):
state = frozenset((player1_choice, player2_choice))
return outcomes[state]
This version of play()
is incredibly simple. Just two lines of code! You could even write it as a single line if you wanted to:
def play(player1_choice, player2_choice):
return outcomes[frozenset((player1_choice, player2_choice))]
Personally, I prefer the two-line version over the single-line version.
Your new play()
function follows the rules of the game and is commutative:
>>> play("rock", "paper")
'paper wins'
>>> play("paper", "rock")
'paper wins'
play()
even raises a KeyError
if it gets called with an invalid choice, but the error is less helpful now that the keys of the outcomes
dictionary are sets:
>>> play("lizard", "paper")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 21, in play
return outcomes[state]
KeyError: frozenset({'lizard', 'paper'})
The vague error would likely not be an issue, however. In this article, you're only implementing the play()
function. In a true implementation of "Rock Paper Scissors," you'd most likely capture user input and validate that before ever passing the user's choice to play()
.
So, how much faster is this implementation versus the previous ones? Here are some timing results to compare the performance of the various implementations using IPython's %timeit
magic function. play1()
is the version of play()
from the Advanced Solution #2 section, and play2()
is the current version:
In [1]: %timeit play1("rock", "paper")
141 ns ± 0.0828 ns per loop (mean ± std. dev. of 7 runs, 10,000,000 loops each)
In [2]: %timeit play2("rock", "paper")
188 ns ± 0.0944 ns per loop (mean ± std. dev. of 7 runs, 10,000,000 loops each)
In this case, the solution using the results table is actually slower than the previous implementation. The culprit here is the line that converts the function arguments to a frozenset
. So, although dictionary lookups are fast, and building a table that maps states to outcomes can potentially improve performance, you need to be careful to avoid expensive operations that may end up negating whatever gains you expect to get.
I wrote this article as an exercise. I was curious to know how I'd approach a beginner project like "Rock Paper Scissors" in Python now that I have a lot of experience. I hope you found it interesting. If you have any inkling of inspiration now to revisit some of your own beginner projects, then I think I've done my job!
If you do revisit some of your own beginner projects, or if you've done so in the past, let me know how it went in the comments. Did you learn anything new? How different is your new solution from the one you wrote as a beginner?
An acquaintance from the Julia world, Miguel Raz Guzmán Macedo, turned me on to a blog post by Mosè Giordano. Mosè leverages Julia's multiple dispatch paradigm to write "Rock Paper Scissors" in less than ten lines of code:
I won't get into the details of how Mosè's code works. Python doesn't even support multiple dispatch out-of-the-box. (Although you can use it with some help from the plum
package.)
Mosè's article got my mental gears spinning and encouraged me to revisit "Rock Paper Scissors" in Python to think about how I could approach the project differently.
As I was working through the solution, however, I was reminded of an article I did a reviewed for Real Python quite some time ago:
It turns out the first two solutions I "invented" here are similar to the solution that Chris Wilkerson, the author of Real Python's article, came up with.
Chris's solution is more full-featured. It includes an interactive gameplay mechanism and even uses Python's Enum
type to represent game items. That must have also been where I first heard of "Rock Paper Scissors Lizard Spock."
Floating-point numbers are a fast and efficient way to store and work with numbers, but they come with a range of pitfalls that have surely stumped many fledgling programmers — perhaps some experienced programmers, too! The classic example demonstrating the pitfalls of floats goes like this:
>>> 0.1 + 0.2 == 0.3
False
Seeing this for the first time can be disorienting. But don't throw your computer in the trash bin. This behavior is correct!
This article will show you why floating-point errors like the one above are common, why they make sense, and what you can do to deal with them in Python.
You've seen that 0.1 + 0.2
is not equal to 0.3
but the madness doesn't stop there. Here are some more confounding examples:
>>> 0.2 + 0.2 + 0.2 == 0.6
False
>>> 1.3 + 2.0 == 3.3
False
>>> 1.2 + 2.4 + 3.6 == 7.2
False
The issue isn't restricted to equality comparisons, either:
>>> 0.1 + 0.2 <= 0.3
False
>>> 10.4 + 20.8 > 31.2
True
>>> 0.8 - 0.1 > 0.7
True
So what's going on? Is your computer lying to you? It sure looks like it, but there's more going on beneath the surface.
When you type the number 0.1
into the Python interpreter, it gets stored in memory as a floating-point number. There's a conversion that takes place when this happens. 0.1
is a decimal in base 10, but floating-point numbers are stored in binary. In other words, 0.1
gets converted from base 10 to base 2.
The resulting binary number may not accurately represent the original base 10 number. 0.1
is one example. The binary representation is \(0.0\overline{0011}\). That is, 0.1
is an infinitely repeating decimal when written in base 2. The same thing happens when you write the fraction ⅓ as a decimal in base 10. You end up with the infinitely repeating decimal \(0.\overline{33}\).
Computer memory is finite, so the infinitely repeating binary fraction representation of 0.1
gets rounded to a finite fraction. The value of this number depends on your computer's architecture (32-bit vs. 64-bit). One way to see the floating-point value that gets stored for 0.1
is to use the .as_integer_ratio()
method for floats to get the numerator and denominator of the floating-point representation:
>>> numerator, denominator = (0.1).as_integer_ratio()
>>> f"0.1 ≈ {numerator} / {denominator}"
'0.1 ≈ 3602879701896397 / 36028797018963968'
Now use format()
to show the fraction accurate to 55 decimal places:
>>> format(numerator / denominator, ".55f")
'0.1000000000000000055511151231257827021181583404541015625'
So 0.1
gets rounded to a number slightly larger than its true value.
.as_integer_ratio()
in my article 3 Things You Might Not Know About Numbers in Python.This error, known as floating-point representation error, happens way more often than you might realize.
One email, every Saturday, with one actionable tip.
Always less than 5 minutes of your time.
There are three reasons that a number gets rounded when represented as a floating-point number:
64-bit floating-point numbers are good for about 16 or 17 significant digits. Any number with more significant digits gets rounded. Irrational numbers, like π and e, can't be represented by any terminating fraction in any integer base. So again, no matter what, irrational numbers will get rounded when stored as floats.
These two situations create an infinite set of numbers that can't be exactly represented as a floating-point number. But unless you're a chemist dealing with tiny numbers, or a physicist dealing with astronomically large numbers, you're unlikely to run into these problems.
What about non-terminating rational numbers, like 0.1
in base 2? This is where you'll encounter most of your floating-point woes, and thanks to the math that determines whether or not a fraction terminates, you'll brush up against representation error more often than you think.
In base 10, a fraction terminates if its denominator is a product of powers of prime factors of 10. The two prime factors of 10 are 2 and 5, so fractions like ½, ¼, ⅕, ⅛, and ⅒ all terminate, but ⅓, ⅐, and ⅑ do not. In base 2, however, there is only one prime factor: 2. So only fractions whose denominator is a power of 2 terminate. As a result, fractions like ⅓, ⅕, ⅙, ⅐, ⅑, and ⅒ are all non-terminating when expressed in binary.
You can now understand the original example in this article. 0.1
, 0.2
, and 0.3
all get rounded when converted to floating-point numbers:
>>> # -----------vvvv Display with 17 significant digits
>>> format(0.1, ".17g")
'0.10000000000000001'
>>> format(0.2, ".17g")
'0.20000000000000001'
>>> format(0.3, ".17g")
'0.29999999999999999'
When 0.1
and 0.2
are added, the result is a number slightly larger than 0.3
:
>>> 0.1 + 0.2
0.30000000000000004
Since 0.1 + 0.2
is slightly larger than0.3
and 0.3
gets represented by a number slightly smaller than itself, the expression 0.1 + 0.2 == 0.3
evaluates to False
.
0.1 + 0.2
in many different languages over at Erik Wiffin's aptly named website 0.30000000000000004.com.So, how do you deal with floating-point representation errors when comparing floats in Python? The trick is to avoid checking for equality. Never use ==
, >=
, or <=
with floats. Use the math.isclose()
function instead:
>>> import math
>>> math.isclose(0.1 + 0.2, 0.3)
True
math.isclose()
checks if the first argument is acceptably close to the second argument. But what exactly does that mean? The key idea is to examine the distance between the first argument and the second argument, which is equivalent to the absolute value of the difference of the values:
>>> a = 0.1 + 0.2
>>> b = 0.3
>>> abs(a - b)
5.551115123125783e-17
If abs(a - b)
is smaller than some percentage of the larger of a
or b
, then a
is considered sufficiently close to b
to be "equal" to b
. This percentage is called the relative tolerance. You can specify the relative tolerance with the rel_tol
keyword argument of math.isclose()
which defaults to 1e-9
. In other words, if abs(a - b)
is less than 1e-9 * max(abs(a), abs(b))
, then a
and b
are considered "close" to each other. This guarantees that a
and b
are equal to about nine decimal places.
You can change the relative tolerance if you need to:
>>> math.isclose(0.1 + 0.2, 0.3, rel_tol=1e-20)
False
Of course, the relative tolerance depends on constraints set by the problem you're solving. For most everyday applications, however, the default relative tolerance should suffice.
There's a problem if one of a
or b
is zero and rel_tol
is less than one, however. In that case, no matter how close the nonzero value is to zero, the relative tolerance guarantees that the check for closeness will always fail. In this case, using an absolute tolerance works as a fallback:
>>> # Relative check fails!
>>> # ---------------vvvv Relative tolerance
>>> # ----------------------vvvvv max(0, 1e-10)
>>> abs(0 - 1e-10) < 1e-9 * 1e-10
False
>>> # Absolute check works!
>>> # ---------------vvvv Absolute tolerance
>>> abs(0 - 1e-10) < 1e-9
True
math.isclose()
will do this check for you automatically. The abs_tol
keyword argument determines the absolute tolerance. However, abs_tol
defaults to 0.0
So you'll need to set this manually if you need to check how close a value is to zero.
All in all, math.isclose()
returns the result of the following comparison, which combines the relative and absolute tests into a single expression:
abs(a - b) <= max(rel_tol * max(abs(a), abs(b)), abs_tol)
math.isclose()
was introduced in PEP 485 and has been available since Python 3.5.
One email, every Saturday, to challenge your skills and inspire curiosity.
Always less than 5 minutes of your time.
math.isclose()
?In general, you should use math.isclose()
whenever you need to compare floating-point values. Replace ==
with math.isclose()
:
>>> # Don't do this:
>>> 0.1 + 0.2 == 0.3
False
>>> # Do this instead:
>>> math.isclose(0.1 + 0.2, 0.3)
True
You also need to be careful with >=
and <=
comparisons. Handle the equality separately using math.isclose()
and then check the strict comparison:
>>> a, b, c = 0.1, 0.2, 0.3
>>> # Don't do this:
>>> a + b <= c
False
>>> # Do this instead:
>>> math.isclose(a + b, c) or (a + b < c)
True
Various alternatives to math.isclose()
exist. If you use NumPy, you can leverage numpy.allclose()
and numpy.isclose()
:
>>> import numpy as np
>>> # Use numpy.allclose() to check if two arrays are equal
>>> # to each other within a tolerance.
>>> np.allclose([1e10, 1e-7], [1.00001e10, 1e-8])
False
>>> np.allclose([1e10, 1e-8], [1.00001e10, 1e-9])
True
>>> # Use numpy.isclose() to check if the elements of two arrays
>>> # are equal to each other within a tolerance
>>> np.isclose([1e10, 1e-7], [1.00001e10, 1e-8])
array([ True, False])
>>> np.isclose([1e10, 1e-8], [1.00001e10, 1e-9])
array([ True, True])
Keep in mind that the default relative and absolute tolerances are not the same as math.isclose()
. The default relative tolerance for both numpy.allclose()
and numpy.isclose()
is 1e-05
and the default absolute tolerance for both is 1e-08
.
math.isclose()
is especially useful for unit tests, although there are some alternatives. Python's built-in unittest
module has a unittest.TestCase.assertAlmostEqual()
method. However, that method only uses an absolute difference test. It's also an assertion, meaning that failures raise an AssertionError
, making it unsuitable for comparisons in your business logic.
A great alternative to math.isclose()
for unit testing is the pytest.approx()
function from the pytest
package. Unlike math.isclose()
, pytest.approx()
only takes one argument — namely, the value you expect:
>>> import pytest
>>> 0.1 + 0.2 == pytest.approx(0.3)
True
pytest.approx()
has rel_tol
and abs_tol
keyword arguments for setting the relative and absolute tolerances. The default values are different from math.isclose()
, however. rel_tol
has a default value of 1e-6
and abs_tol
has a default value of 1e-12
.
If the argument passed to pytest.approx()
is array-like, meaning it's a Python iterable like a list or a tuple, or even a NumPy array, then pytest.approx()
behaves similar to numpy.allclose()
and returns whether or not the two arrays are equal within the tolerances:
>>> import numpy as np
>>> np.array([0.1, 0.2]) + np.array([0.2, 0.4]) == pytest.approx(np.array([0.3, 0.6]))
True
pytest.approx()
will even work with dictionary values:
>>> {'a': 0.1 + 0.2, 'b': 0.2 + 0.4} == pytest.approx({'a': 0.3, 'b': 0.6})
True
Floating-point numbers are great for working with numbers whenever absolute precision isn't needed. They are fast and memory efficient. But if you do need precision, then there are some alternatives to floats that you should consider.
There are two built-in numeric types in Python that offer full precision for situations where floats are inadequate: Decimal
and Fraction
.
Decimal
TypeThe Decimal
type can store decimal values exactly with as much precision as you need. By default, Decimal
preserves 28 significant figures, but you can change this to whatever you need to suit the specific problem you're solving:
>>> # Import the Decimal type from the decimal module
>>> from decimal import Decimal
>>> # Values are represented exactly so no rounding error occurs
>>> Decimal("0.1") + Decimal("0.2") == Decimal("0.3")
True
>>> # By default 28 significant figures are preserved
>>> Decimal(1) / Decimal(7)
Decimal('0.1428571428571428571428571429')
>>> # You can change the significant figures if needed
>>> from decimal import getcontext
>>> getcontext().prec = 6 # Use 6 significant figures
>>> Decimal(1) / Decimal(7)
Decimal('0.142857')
You can read more about the Decimal
type in the Python docs.
Fraction
TypeAnother alternative to floating-point numbers is the Fraction
type. Fraction
can store rational numbers exactly and overcomes representation error issues encountered by floating-point numbers:
>>> # import the Fraction type from the fractions module
>>> from fractions import Fraction
>>> # Instantiate a Fraction with a numerator and denominator
>>> Fraction(1, 10)
Fraction(1, 10)
>>> # Values are represented exactly so no rounding error occurs
>>> Fraction(1, 10) + Fraction(2, 10) == Fraction(3, 10)
True
Both Fraction
and Decimal
offer numerous benefits over standard floating-point values. However, these benefits come at a price: reduced speed and higher memory consumption. If you don't need absolute precision, you're better off sticking with floats. But for things like financial and mission-critical applications, the tradeoffs incurred by Fraction
and Decimal
may be worthwhile.
Floating-point values are both a blessing and a curse. They offer fast arithmetic operations and efficient memory use at the cost of inaccurate representation. In this article, you learned:
Fraction
and Decimal
typesIf you learned something new, then there might be even more that you don't know about numbers in Python. For example, did you know the int
type isn't the only integer type in Python? Find out what the other integer type is and other little-known facts about numbers in my article 3 Things You Might Not Know About Numbers in Python.
Thanks to Brian Okken for helping catch an issue with one of the pytest.approx()
examples.
A common question I’ve seen from beginning Python programmers is, “How do I make my code more Pythonic?” The problem with a word like “Pythonic” is that its meaning is nebulous: it means different things to different people.
The meaning isn’t static, either. Whether or not code is Pythonic can depend on which version of Python you’re using, and best practices for writing Pythonic code may change over time.
In this article, I’ll share my perspective on what makes code Pythonic by looking at a few concrete examples. I’ll also leave you with some hand-picked resources that will help you build a mental model for deciding when code is Pythonic or not.
But first, let’s agree on at least some kind of definition for the word Pythonic.
The Python language is over 30 years old. In that time, Python programmers have collectively gained an enormous amount of experience using the language for a wide range of purposes. Over time, that collective experience has been shared and distilled into best practices — commonly referred to as the Pythonic way.
The Zen of Python, written by Tim Peters and accessible from any Python installation by typing import this
into the REPL, traditionally exemplifies the Pythonic mindset:
>>> import this
The Zen of Python, by Tim Peters
Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!
The beauty of the Zen of Python is also the most annoying feature for Python beginners. The Zen elegantly captures the spirit of what it means to be Pythonic without giving any explicit advice. For example, consider the first principle: “Beautiful is better than ugly.” OK, sure! But how do I take my ugly code and make it beautiful? What even is beautiful code in the first place?
The ambiguity of the Zen of Python, however frustrating, is what makes it as relevant now as when Tim Peters wrote it in 1999. It serves as a set of guiding principles that equip you with a sense for distinguishing Pythonic code from un-Pythonic code and provides a foundation for a mental framework for making your own decisions.
So, where does this leave us concerning an actual definition of the word “Pythonic?” The best definition that I’ve found is from a 2014 StackOverflow answer to the question “What does Pythonic mean?” that describes Pythonic code as:
[C]ode that doesn’t just get the syntax right but that follows the conventions of the Python community and uses the language in the way it is intended to be used.
There are two key takeaways here:
So now that we have at least some understanding of what Python programmers mean when they refer to code as Pythonic let’s look at three specific ways you can write more Pythonic code right now.
One email, every Saturday, with one actionable tip.
Always less than 5 minutes of your time.
PEP8 is Python’s official style guide. PEP stands for Python Enhancement Proposal. PEPs are documents that propose new Python features and serve as official documentation for the feature while the Python community debates its acceptance or rejection. Following PEP8 won’t quite get your code to Pythonic perfection, but it does go a long way towards making your code look familiar to many Python programmers.
PEP8 deals with things like how to handle whitespace in your code, such as using four spaces for indentation instead of a tab character, or what the maximum line length should be, which, according to PEP8, is 79 characters — although this is probably the most widely ignored PEP8 recommendation.
If you’re new to Python programming, one of the first things I recommend internalizing from PEP8 is the recommendations for naming conventions. For example, you should write function and variable names in the lowercase_with_underscores
style:
# Correct
seconds_per_hour = 3600
# Incorrect
secondsperhour = 3600
secondsPerHour = 3600
Class names should use the CapitalizedWords
style:
# Correct
class SomeThing:
pass
# Incorrect
class something:
pass
class some_thing:
pass
Write constants in the UPPER_CASE_WITH_UNDERSCORES
style:
# Correct
PLANCK_CONSTANT = 6.62607015e-34
# Incorrect
planck_constant = 6.6260715e-34
planckConstant = 6.6260715e-34
The whitespace recommendations laid out in PEP8 include how to use spaces around operators, around function parameter names and arguments, and how to break long lines. While years of practicing reading and writing PEP8 compliant Python code will help you internalize these recommendations, it’s still a lot to remember.
Don’t worry if you can’t memorize all of PEP8’s conventions. You don’t need to! Tools like flake8
can help you find and fix PEP8 issues in your code. You can install flake8
with pip
:
# Linux/macOS
$ python3 -m pip install flake8
# Windows
$ python -m pip install flake8
flake8
can be used as a command-line application to scan a Python file for style violations. For example, let’s say I have a text file called myscript.py
containing the following code:
def add( x, y ):
return x+y
num1=1
num2=2
print( add(num1,num2) )
Running flake8
against this code tells you what violations there are and exactly where they’re located:
$ flake8 myscript.py
myscript.py:1:9: E201 whitespace after '('
myscript.py:1:11: E231 missing whitespace after ','
myscript.py:1:13: E202 whitespace before ')'
myscript.py:4:1: E305 expected 2 blank lines after class or function definition, found 1
myscript.py:4:5: E225 missing whitespace around operator
myscript.py:5:5: E225 missing whitespace around operator
myscript.py:6:7: E201 whitespace after '('
myscript.py:6:16: E231 missing whitespace after ','
myscript.py:6:22: E202 whitespace before ')'
Each line of output from flake8
tells you which file the problem is in, which line the problem is on, which column in the line the error starts on, an error number (use these codes to configure flake8
to ignore specific errors if you wish), as well as a description of the error:
You can even set up editors like VS Code to lint your code with flake8
while you write it to continuously check your code for PEP8 violations. When flake8
finds an issue, a squiggly red line appears underneath the offending portion of your code, and you can see which errors have been detected in the Problems tab of the built-in terminal:
flake8
is an excellent tool for finding PEP8 errors in your code, but you still have to manually fix all of those errors. This can be a lot of work. Fortunately, there’s a way to automate the whole process.
The black
auto-formatter for Python is a tool for automatically formatting your code to conform to PEP8. Of course, PEP8 recommendations leave a lot of wiggle room for stylistic choices and black
makes a lot of decisions for you. You may or may not agree with these decisions. black
is minimally configurable, so you may want to play around with it first before committing to using it.
You can install black
with pip
:
# Linux/macOS
$ python3 -m pip install black
# Windows
$ python -m pip install black
Once installed, you can use the black --check
command in your shell to see if black
would make any changes to a file:
$ black --check myscript.py
would reformat myscript.py
Oh no! 💥 💔 💥
1 file would be reformatted.
You can use the --diff
flag to see a diff of what changes black
would make:
$ black --diff myscript.py
--- myscript.py 2022-03-15 21:27:20.674809 +0000
+++ myscript.py 2022-03-15 21:28:27.357107 +0000
@@ -1,6 +1,7 @@
-def add( x, y ):
- return x+y
+def add(x, y):
+ return x + y
-num1=1
-num2=2
-print( add(num1,num2) )
+
+num1 = 1
+num2 = 2
+print(add(num1, num2))
would reformat myscript.py
All done! ✨ 🍰 ✨
1 file would be reformatted.
To automatically format your file, pass the file name to the black
command:
$ black myscript.py
reformatted myscript.py
All done! ✨ 🍰 ✨
1 file reformatted.
# Show the formatted file
$ cat myscript.py
def add(x, y):
return x + y
num1 = 1
num2 = 2
print(add(num1, num2))
To check that your file is PEP8 compliant now, run flake8
against it again and see if you get any errors:
# No output from flake8 so everything is good!
$ flake8 myscript.py
One thing to keep in mind when using black
is that, by default, black
sets the maximum line length to 88 columns. This diverges from PEP8’s recommendation for 79 columns lines, so you may see flake8
report line length errors even when using black
. You can configure black
to use 79 columns or configure flake8
to accept longer line lengths. Many Python devs use 88 columns instead of 79, and some even set black
and flake8
to use even longer line lengths.
It’s important to remember that PEP8 is just a set of recommendations, although these recommendations are taken seriously by many Python programmers. But there’s nothing in Python that enforces the PEP8 style guide. If there’s something in PEP8 that you strongly disagree with, then, by all means, ignore it! If you do want to adhere strictly to PEP8, however, tools like flake8
and black
can make your life a lot easier.
In languages like C or C++, keeping track of an index variable while looping over an array is common. For example, when asked to print the elements of a list, it’s not uncommon for new Python programmers coming from C or C++ to write something like the following:
>>> names = ["JL", "Raffi", "Agnes", "Rios", "Elnor"]
>>> # Using a `while` loop
>>> i = 0
>>> while i < len(names):
... print(names[i])
... i += 1
JL
Raffi
Agnes
Rios
Elnor
>>> # Using a `for` loop
>>> for i in range(len(names)):
... print(names[i])
JL
Raffi
Agnes
Rios
Elnor
Instead of iterating over an index, however, you can iterate over items in a list directly:
>>> for name in names:
... print(name)
JL
Raffi
Agnes
Rios
Elnor
However, avoiding C-style loops goes a lot deeper than just directly iterating over items in a list. Leveraging Python idioms, such as list comprehensions, built-in functions like min()
, max()
, and sum()
and making use of object methods can help take your Python code to the next level.
One email, every Saturday, to challenge your skills and inspire curiosity.
Always less than 5 minutes of your time.
for
LoopsA common programming task is to process the elements from one array and store the results in a new array. For example, suppose you have a list of numbers and want to transform it into a list of the squares of those numbers. You know that you should avoid C-style loops, so you may end up writing something like this:
>>> nums = [1, 2, 3, 4, 5]
>>> squares = []
>>> for num in nums:
... squares.append(num ** 2)
...
>>> squares
[1, 4, 9, 16, 25]
A more Pythonic way to do this is to use a list comprehension:
>>> squares = [num ** 2 for num in nums] # <-- List comprehension
>>> squares
[1, 4, 9, 16, 25]
List comprehensions can be difficult to grok at first. However, if you’re familiar with set-builder notation for writing sets in mathematics, then list comprehensions may already look familiar.
Here’s how I usually think about list comprehensions:
[]
.append()
method if you were building the list using a for
loop:[num ** 2]
for
loop’s header at the end of the list:[num ** 2 for num in nums]
List comprehensions are an important concept to master when writing Pythonic code. But they can be overused. They're also not the only kind of comprehension in Python. In the following sections, you'll learn about other comprehensions, such as generator expressions and dictionary comprehensions, and see an example of when it makes sense to avoid a list comprehension.
min()
, max()
, and sum()
Another common programming task is finding the minimum or maximum value in an array of numbers. Using a for
loop, you can find the minimum number in a list as follows:
>>> nums = [10, 21, 7, -2, -5, 13]
>>> min_value = nums[0]
>>> for num in nums[1:]:
... if num < min_value:
... min_value = num
...
>>> min_value
-5
A more Pythonic way to do this is to use the min()
built-in function:
>>> min(nums)
-5
In a similar vein, there’s no need to write a loop to find the maximum value in a list. You can use the max()
built-in function:
>>> max(nums)
21
To find the sum of the numbers in a list, you could write a for
loop. But a more Pythonic approach is to use the sum()
function:
>>> # Not Pythonic: Use a `for` loop
>>> sum_of_nums = 0
>>> for num in nums:
... sum_of_nums += num
...
>>> sum_of_nums
44
>>> # Pythonic: Use `sum()`
>>> sum(nums)
44
Another Pythonic use of sum()
is to count the number of elements of a list for which some condition holds. For example, here’s a for
loop that counts the number of strings in a list that start with the letter A:
>>> capitals = ["Atlanta", "Houston", "Denver", "Augusta"]
>>> count_a_capitals = 0
>>> for capital in capitals:
... if capital.startswith("A"):
... count_a_capitals += 1
...
>>> count_a_capitals
2
Combining sum()
with a list comprehension reduces the for
loop to a single line of code:
>>> sum([capital.startswith("A") for capital in capitals])
2
As lovely as that is, you can make it even more Pythonic by replacing the list comprehension with a generator expression by removing the brackets around the list:
>>> sum(capital.startswith("A") for capital in capitals)
2
How exactly does this work? Both the list comprehension and the generator expression return an iterable containing True
and False
values corresponding to whether or not the string in the capitals
list starts with the letter "A"
:
>>> [capital.startswith("A") for capital in capitals]
[True, False, False, True]
In Python, True
and False
are integers in disguise. True
is equal to 1
and False
is equal to 0
:
>>> isinstance(True, int)
True
>>> True == 1
True
>>> isinstance(False, int)
True
>>> False == 0
True
When you pass the list comprehension or generator expression to sum()
, the True
and False
values get treated like 1
and 0
, respectively. Since there are two True
values and two False
values, the total sum is equal to 2
.
Using sum()
to count how many list elements satisfy a condition highlights an important point about the concept of Pythonic code. Personally, I find this use of sum()
to be very Pythonic. After all, it leverages several Python language features to create what is, in my opinion, concise-yet-readable code. However, not every Python developer may agree with me.
One could argue that this example violates one of the principles of the Zen of Python: “Explicit is better than implicit.” After all, it’s not obvious that True
and False
are integers and that sum()
should even work with a list of True
and False
values. Understanding this use of sum()
requires a deep understanding of Python’s built-in types.
True
and False
as integers, as well as other surprising facts about numbers in Python, check out my article 3 Things You Might Not Know About Numbers in Python.There is no set of rigid rules that tell you whether or not code is Pythonic. There’s always a gray area. Use your best judgment when confronted with a code example that feels like it might be in this gray area. Always err on the side of readability, and don’t be afraid to reach out to coworkers or use social media to get help.
A big part of writing clean, Pythonic code boils down to picking the proper data structure for the task at hand. Python is well-known as a “batteries included” language. Several of the batteries included with Python are efficient and ready-to-use data structures.
Suppose you have a CSV file called clients.csv
containing client data for a business that looks something like this:
first_name,last_name,email,phone
Manuel,Wilson,mwilson@example.net,757-942-0588
Stephanie,Gonzales,sellis@example.com,385-474-4769
Cory,Ali,coryali17@example.net,810-361-3885
Adam,Soto,adams23@example.com,724-603-5463
Let’s say you’re tasked with writing a program that takes an email address as input and outputs the phone number of the client with that email if such a client exists. How would you go about doing it?
You can read each row of this file as a dictionary using the DictReader
object from the csv
module:
>>> import csv
>>> with open("clients.csv", "r") as csvfile:
... clients = list(csv.DictReader(csvfile))
...
>>> clients
[{'first_name': 'Manuel', 'last_name': 'Wilson', 'email': 'mwilson@example.net', 'phone': '757-942-0588'},
{'first_name': 'Stephanie', 'last_name': 'Gonzales', 'email': 'sellis@example.com', 'phone': '385-474-4769'},
{'first_name': 'Cory', 'last_name': 'Ali', 'email': 'coryali17@example.net', 'phone': '810-361-3885'},
{'first_name': 'Adam', 'last_name': 'Soto', 'email': 'adams23@example.com', 'phone': '724-603-5463'}]
clients
is a list of dictionaries, so to find the client with a given email, say sellis@example.com
, you’ll need to loop over the list and compare each client’s email with the target email until the right client is found:
>>> target = "sellis@example.com"
>>> phone = None
>>> for client in clients:
... if client["email"] == target:
... phone = client["phone"]
... break
...
>>> print(phone)
385-474-4769
The problem with this code is that looping over the list of clients is inefficient. If there are a large number of clients in the CSV file, your program could be spending a significant amount of time scanning the list for a client with a matching email. If you need to do this check often, this could result in a whole bunch of wasted time.
A more Pythonic approach is to forget about storing the clients in a list and use a dictionary to map clients to their email addresses. A great way to do this is with a dictionary comprehension:
>>> with open("clients.csv", "r") as csvfile:
... # Use a `dict` comprehension instead of a `list`
... clients = {row["email"]: row["phone"] for row in csv.DictReader(csvfile)}
...
>>> clients
{'mwilson@example.net': '757-942-0588', 'sellis@example.com': '385-474-4769',
'coryali17@example.net': '810-361-3885', 'adams23@example.com': '724-603-5463'}
Dictionary comprehensions are a lot like list comprehensions:
{}
{row[“email”]: row[“phone”]}
for
expression that loops over each row in the CSV file:{row[“email”]: row[“phone”] for row in csv.DictReader(csvfile)}
Translated into a for
loop, this dictionary comprehension would look something like this:
>>> clients = {}
>>> with open("clients.csv", "r") as csvfile:
... for row in csv.DictReader(csvfile):
... clients[row["email"]] = row["phone"]
With the clients
dictionary made, you can find a client’s phone number using their email address without having to write any more loops:
>>> target = "sellis@example.com"
>>> clients[target]
385-474-4769
This code is not only shorter than looping over a list, it’s much more efficient. Python can jump straight to the correct value in the dictionary without any loops. There’s a problem, however. If no client in clients
has a matching email, then a KeyError
will be raised:
>>> clients["tsanchez@example.com"]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
KeyError: 'tsanchez@example.com'
One way to handle this is to catch the KeyError
and print a default value if no client is found:
>>> target = "tsanchez@example.com"
>>> try:
... phone = clients[target]
... except KeyError:
... phone = None
...
>>> print(phone)
None
There’s a more Pythonic way to do this, though, using the dictionary’s .get()
method. .get()
returns a key’s corresponding value if the key exists and None
otherwise:
>>> clients.get("sellis@example.com")
'385-474-4769'
Let’s compare the two solutions side-by-side:
import csv
target = "sellis@example.com"
phone = None
# Un-Pythonic: loop over a list
with open("clients.csv", "r") as csvfile:
clients = list(csv.DictReader(csvfile))
for client in clients:
if client["email"] == target:
phone = client["phone"]
break
print(phone)
# Pythonic: lookup in a dictionary
with open("clients.csv", "r") as csvfile:
clients = {row["email"]: row["phone"] for row in csv.DictReader(csvfile)}
phone = clients.get(target)
print(phone)
The Pythonic solution is more concise and efficient without sacrificing readability.
Sets are an undervalued data structure in Python. As a result, even intermediate Python developers tend to ignore sets and miss out on opportunities to use them to their advantage.
Perhaps the most well-known use-case for sets in Python is to remove duplicates from a list:
>>> nums = [1, 3, 2, 3, 1, 2, 3, 1, 2]
>>> unique_nums = list(set(nums))
>>> unique_nums
[1, 2, 3]
But there’s so much more that you can do with sets. One use-case I’ve used often in my code is to use sets to filter values from an iterable efficiently. This works best when you also need unique values.
Here’s a contrived but not unrealistic example. Suppose a shop owner has a CSV file of clients containing their email addresses. We’ll reuse the clients.csv
file from the previous section. The shop owner has another CSV file of orders from the last month that also contains email addresses. Maybe this CSV file is called orders.csv
and looks something like this:
date,email,items_ordered
2022/03/01,adams23@example.net,2
2022/03/04,sellis@example.com,3
2022/03/07,adams23@example.net,1
The shop owner would like to email every client who didn’t order anything from the past month with a discount coupon. One way to do this would be to read the emails from the clients.csv
and orders.csv
files and use a list comprehension to filter the client’s emails:
>>> import csv
>>> # Create a list of all client emails
>>> with open("clients.csv", "r") as clients_csv:
... client_emails = [row["email"] for row in csv.DictReader(clients_csv)]
...
>>> # Create a list of emails from orders
>>> with open("orders.csv") as orders_csv:
... order_emails = [row["email"] for row in csv.DictReader(orders_csv)]
...
>>> # Use a list comprehension to filter the clients emails
>>> coupon_emails = [email for email in clients_emails if email not in order_emails]
>>> coupon_emails
["mwilson@example.net", "coryali17@example.net"]
The above code works fine and certainly looks Pythonic. But suppose the shop owner has millions of clients and orders each month. (They’re apparently very successful!) Filtering the emails to determine which customers to send coupons to requires looping over the entire client_emails
list. And what if there are duplicate rows in the client.csv
and orders.csv
files? Accidents happen, you know.
A more Pythonic approach would be to read in the client and order emails into sets and use the set difference operator to filter the set of client emails:
>>> import csv
>>> # Create a set of all client emails using a set comprehension
>>> with open("clients.csv", "r") as clients_csv:
... client_emails = {row["email"] for row in csv.DictReader(clients_csv)}
...
>>> # Create a set of emails frp, orders using a set comprehension
>>> with open("orders.csv", "r") as orders_csv:
... order_emails = {row["email"] for row in csv.DictReader(orders_csv)}
...
>>> # Filter the client emails using set difference
>>> coupon_emails = client_emails - order_emails
>>> coupon_emails
{"mwilson@example.net", "coryali17@example.net"}
This approach is much more efficient than the previous one because it only loops over the client emails once, not twice. It also has the advantage of naturally removing any duplicate emails from both CSV files.
You can’t learn to write clean Pythonic code overnight. You need to study lots of code examples, practice writing your own code, and consult with other Python developers. To help you on your journey, I’ve compiled a list of three books that I’ve found immensely helpful for grokking the Pythonic way.
Disclaimer: The following sections contain affiliate links. If you decide to purchase one of the books through my link, I will receive a small commission at no cost to you.
Dan Bader's short and sweet book Python Tricks: A Buffet of Awesome Python Features is an excellent starting place for beginner-to-intermediate Python programmers to learn more about writing Pythonic code.
Python Tricks will teach you patterns for writing clean, idiomatic Python, best practices for writing functions, how to use Python's object-oriented programming features effectively, and a whole lot more.
Brett Slatkin's Effective Python was the first book I read after learning the Python syntax that opened my eyes to the power of idiomatic Pythonic code.
As the book's subtitle states, Effective Python covers 90 specific ways to write better Python. The first chapter alone, titled Python Thinking, is a goldmine of tips and tricks that even beginner Python programmers will find helpful, although beginners may find the rest of the book difficult to follow.
If I could only own one book about Python, Luciano Ramalho's Fluent Python would be the one.
Full of practical examples supported by clear exposition, Fluent Python is an excellent guide for anyone looking to learn how to write Pythonic code. However, keep in mind that Fluent Python is not for beginning Python programmers. As stated in the book's preface:
If you are just learning Python, this book is going to be hard to follow. Not only that, if you read it too early in your Python journey, it may give you the impression that every Python script should leverage special methods and metaprogramming tricks. Premature abstraction is as bad as premature optimization.
Experienced Python programmers will benefit greatly from the book, however.
Ramalho recently updated his book for modern Python. Currently, the second edition is only available for pre-order. I strongly recommend pre-ordering the second edition as the first edition is now outdated.
This article covered a lot of ground. You learned:
These tips will help you write more Pythonic code, but they're just a start. Mastering Python takes years. During your years of working towards Python mastery, accepted norms for Pythonic code may change, so it's crucial to stay up-to-date with the current best practices. The r/learnpython subreddit can be a good place to ask questions and get help. I'm also always happy to answer questions on Twitter.
But the first step is to get your hands dirty and practice what you've learned. As the Zen of Python says: "Now is better than never."
What do you think is the output of the following Python code?
>>> flag = "🇺🇸"
>>> reversed_flag = flag[::-1]
>>> print(reversed_flag)
Questions like this make me want to immediately open a Python REPL and try the code out because I think I know what the answer is, but I'm not very confident in that answer.
Here's my line of thinking when I first saw this question:
flag
string contains a single character.[::-1]
slice reverses the flag
string.reversed_flag
must be "🇺🇸"
.That's a perfectly valid argument. But is the conclusion true? Take a look:
>>> flag = "🇺🇸"
>>> reversed_flag = flag[::-1]
>>> print(reversed_flag)
🇸🇺
What in the world is going on here?
"🇺🇸"
Really Contain a Single Character?When the conclusion of a valid argument is false, one of its premises must be false, too. Let's start from the top:
The flag
string contains a single character.
Is that so? How can you tell how many characters a string has?
In Python, you can use the built-in len()
function to get the total number of characters in a string:
>>> len("🇺🇸")
2
Oh.
That's weird. You can only see a single thing in the string "🇺🇸"
— namely the US flag — but a length of 2
jives with the result of flag[::-1]
. Since the reverse of "🇺🇸"
is "🇸🇺"
, this seems to imply that somehow "🇺🇸" == "🇺 🇸"
.
There are a few different ways that you can see all of the characters in a string using Python:
>>> # Convert a string to a list
>>> list("🇺🇸")
['🇺', '🇸']
>>> # Loop over each character and print
>>> for character in "🇺🇸":
... print(character)
...
🇺
🇸
The US flag emoji isn’t the only flag emoji with two characters:
>>> list("🇿🇼") # Zimbabwe
['🇿', '🇼']
>>> list("🇳🇴") # Norway
['🇳', '🇴']
>>> list("🇨🇺") # Cuba
['🇨', '🇺']
>>> # What do you notice?
And then there’s the Scottish flag:
>>> list("🏴")
['🏴', '\U000e0067', '\U000e0062', '\U000e0073', '\U000e0063',
'\U000e0074', '\U000e007f']
OK, what is that all about?
The unnerving thing about these examples is that they imply that you can't tell what characters are in a string just by looking at your screen.
Or, perhaps more deeply, it makes you question your understanding of the term character.
The term character in computer science can be confusing. It tends to get conflated with the word symbol, which, to be fair, is a synonym for the word character as it's used in English vernacular.
In fact, when I googled character computer science
, the very first result I got was a link to a Technopedia article that defines a character as:
"[A] display unit of information equivalent to one alphabetic letter or symbol."
— Technopedia, "Character (Char)"
That definition seems off, especially in light of the US flag example that indicates that a single symbol may be comprised of at least two characters.
The second Google result I get is Wikipedia. In that article, the definition of a character is a bit more liberal:
"[A] character is a unit of information that roughly corresponds to a grapheme, grapheme-like unit, or symbol, such as in an alphabet or syllabary in the written form of a natural language."
— Wikipedia, "Character (computing)"
Hmm... using the word "roughly" in a definition makes the definition feel, shall I say, non-definitive.
But the Wikipedia article goes on to explain that the term character has been used historically to "denote a specific number of contiguous bits.”
Then, a significant clue to the question about how a string with one symbol can contain two or more characters:
"A character is most commonly assumed to refer to 8 bits (one byte) today... All [symbols] can be represented with one or more 8-bit code units with UTF-8."
— Wikipedia, "Character (computing)”
OK! Maybe things are starting to make a little bit more sense. A character represents a unit of text and is often stored as one byte of information . The symbols that we see in a string can be made up of multiple 8-bit (1 byte) UTF-8 code units.
Characters are not the same as symbols. It seems reasonable now that one symbol could be made up of multiple characters, just like flag emojis.
But what is a UTF-8 code unit?
A little further down the Wikipedia article on characters, there’s a section called Encoding that explains:
"Computers and communication equipment represent characters using a character encoding that assigns each character to something – an integer quantity represented by a sequence of digits, typically – that can be stored or transmitted through a network. Two examples of usual encodings are ASCII and the UTF-8 encoding for Unicode."
— Wikipedia, "Character (computing)"
There’s another mention of UTF-8! But now I need to know what a character encoding is.
According to Wikipedia, a character encoding assigns each character to a number. What does that mean?
Doesn’t it mean that you can pair each character with a number? So, you could do something like pair each uppercase letter in the English alphabet with an integer 0 through 25.
You can represent this pairing using tuples in Python:
>>> pairs = [(0, "A"), (1, "B"), (2, "C"), ..., (25, "Z")]
>>> # I'm omitting several pairs here -----^^^
Stop for a moment and ask yourself: “Can I create a list of tuples like the one above without explicitly writing out each pair?"
One way is to use Python’s enumerate()
function. enumerate()
takes an argument called iterable and returns a tuple containing a count that defaults to 0 and the values obtained from iterating over iterable.
Here’s a look at enumerate()
in action:
>>> letters = "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
>>> enumerated_letters = list(enumerate(letters))
>>> enumerated_letters
[(0, 'A'), (1, 'B'), (2, 'C'), (3, 'D'), (4, 'E'), (5, 'F'), (6, 'G'),
(7, 'H'), (8, 'I'), (9, 'J'), (10, 'K'), (11, 'L'), (12, 'M'), (13, 'N'),
(14, 'O'), (15, 'P'), (16, 'Q'), (17, 'R'), (18, 'S'), (19, 'T'), (20, 'U'),
(21, 'V'), (22, 'W'), (23, 'X'), (24, 'Y'), (25, 'Z')]
There’s an easier way to make all of the letters, too.
Python’s string
module has a variable called ascii_uppercase
that points to a string containing all of the uppercase letters in the English alphabet:
>>> import string
>>> string.ascii_uppercase
'ABCDEFGHIJKLMNOPQRSTUVWXYZ'
>>> enumerated_letters = list(enumerate(string.ascii_uppercase))
>>> enumerated_letters
[(0, 'A'), (1, 'B'), (2, 'C'), (3, 'D'), (4, 'E'), (5, 'F'), (6, 'G'),
(7, 'H'), (8, 'I'), (9, 'J'), (10, 'K'), (11, 'L'), (12, 'M'), (13, 'N'),
(14, 'O'), (15, 'P'), (16, 'Q'), (17, 'R'), (18, 'S'), (19, 'T'),
(20, 'U'), (21, 'V'), (22, 'W'), (23, 'X'), (24, 'Y'), (25, 'Z')]
OK, so we’ve associated characters to integers. That means we’ve got a character encoding!
But, how do you use it?
To encode the string ”PYTHON”
as a sequence of integers, you need a way to look up the integer associated with each character. But, looking things up in a list of tuples is hard. It’s also really inefficient. (Why?)
Dictionaries are good for looking things up. If we convert enumerated_letters
to a dictionary, we can quickly look up the letter associated with an integer:
>>> int_to_char = dict(enumerated_letters)
>>> # Get the character paired with 1
>>> int_to_char[1]
'B'
>>> # Get the character paired with 15
>>> int_to_char[15]
'P'
However, to encode the string ”PYTHON”
you need to be able to look up the integer associated with a character. You need the reverse of int_to_char.
How do you swap keys and values in a Python dictionary?
One way is use the reversed()
function to reverse key-value pairs from the int_to_char
dictionary:
>>> # int_to_char.items() is a "list" of key-value pairs
>>> int_to_char.items()
dict_items([(0, 'A'), (1, 'B'), (2, 'C'), (3, 'D'), (4, 'E'), (5, 'F'),
(6, 'G'), (7, 'H'), (8, 'I'), (9, 'J'), (10, 'K'), (11, 'L'), (12, 'M'),
(13, 'N'), (14, 'O'), (15, 'P'), (16, 'Q'), (17, 'R'), (18, 'S'),
(19, 'T'), (20, 'U'), (21, 'V'), (22, 'W'), (23, 'X'), (24, 'Y'),
(25, 'Z')])
>>> # The reversed() function can reverse a tuple
>>> pair = (0, "A")
>>> tuple(reversed(pair))
('A', 0)
You can write a generator expression that reverses all of the pairs in int_to_char.items()
and use that generator expression to populate a dictionary:
>>> char_to_int = dict(reversed(pair) for pair in int_to_char.items())
>>> # Reverse the pair-^^^^^^^^^^^^^^
>>> # For every key-value pair--------^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>> # Get the integer associated with B
>>> char_to_int["B"]
1
>>> # Get the integer associated with P
>>> char_to_int["P"]
15
It’s good that you paired each letter with a unique integer. Otherwise, this dictionary reversal wouldn’t have worked. (Why?)
Now you can encode strings as list of integers using the char_to_int
dictionary and a list comprehension:
>>> [char_to_int[char] for char in "PYTHON"]
[15, 24, 19, 7, 14, 13]
And you can convert a list of integers into a string of uppercase characters using int_to_char
in a generator expression with Python's string .join()
method:
>>> "".join(int_to_char[num] for num in [7, 4, 11, 11, 14])
'HELLO'
But, there’s a problem.
Your encoding can’t handle strings with things like punctuation, lowercase letters, and whitespace:
>>> [char_to_int[char] for char in "monty python!"]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 1, in <listcomp>
KeyError: 'm'
>>> # ^^^^^^^-----char_to_int has no "m" key
One way to fix this is to create an encoding using a string containing all of the lowercase letters, punctuation marks, and whitespace characters that you need.
But, in Python, there’s almost always a better way. Python’s string
module contains a variable called printable
that gives you a string containing a whole bunch of printable characters:
>>> string.printable
'0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ!"#$%&\'()*+,-./:;<=>?@[\\]^_`{|}~ \t\n\r\x0b\x0c'
Would you have included all of those characters if you were making your own string from scratch?
Now you can make new dictionaries for encoding and decoding the characters in string.printable
:
>>> int_to_printable = dict(enumerate(string.printable))
>>> printable_to_int = dict(reversed(item) for item in int_to_printable.items())
You can use these dictionaries to encode and decode more complicated strings:
>>> # Encode the string "monty python!"
>>> encoded_string = [printable_to_int[char] for char in "monty python!"]
>>> encoded_string
[22, 24, 23, 29, 34, 94, 25, 34, 29, 17, 24, 23, 62]
>>> # Decode the encoded string
>>> decoded_string = "".join(int_to_printable[num] for num in encoded_string)
>>> decoded_string
'monty python!'
You’ve now made two different character encodings! And they really are different. Just look at what happens when you decode the same list of integers using both encodings:
>>> encoded_string = [15, 24, 19, 7, 14, 13]
>>> # Decode using int_to_char (string.ascii_uppercase)
>>> "".join(int_to_char[num] for num in encoded_string)
'PYTHON'
>>> # Decode using int_to_printable(string.printable)
>>> "".join(int_to_printable[num] for num in encoded_string)
'foj7ed'
Not even close!
So, now we know a few things about character encodings:
What does any of this have to do with UTF-8?
Wikipedia's article on characters mentions two different character encodings:
"Two examples of usual encodings are ASCII and the UTF-8 encoding for Unicode."
— Wikipedia, "Character (computing)”
OK, so ASCII and UTF-8 are specific kinds of character encoding.
According to the Wikipedia article on ASCII:
"ASCII was the most common character encoding on the World Wide Web until December 2007, when UTF-8 encoding surpassed it; UTF-8 is backward compatible with ASCII."
— Wikipedia, "ASCII”
UTF-8 isn’t just the dominant character encoding for the web. It’s also the primary character encoding for Linux and macOS operating systems and is even the default for Python code.
In fact, you can see how UTF-8 encodes characters as integers using the .encode()
method on Python string objects. But .encode()
doesn't return a list of integers. Instead, encode()
returns a bytes
object:
>>> encoded_string = "PYTHON".encode()
>>> # The encoded string *looks* like a string still,
>>> # but notice the b in front of the first quote
>>> encoded_string
b'PYTHON'
>>> # b stands for bytes, which is the type of
>>> # object returned by .encode()
>>> type(encoded_string)
<class 'bytes'>
The Python docs describe a bytes
object as “an immutable sequence of integers in the range 0 <= x < 256
.” That seems a little weird considering that the encoded_string
object displays the characters in the string “PYTHON”
and not a bunch of integers.
But let’s accept this and see if we can tease out the integers somehow.
The Python docs say that bytes
is a "sequence," and Python's glossary defines a sequence as “[a]n iterable which supports efficient element access using integer indices.”
So, it sounds like you can index a bytes
object the same way that you can index a Python list
object. Let's try it out:
>>> encoded_string[0]
80
Aha!
What happens when you convert encoded_string
to a list?
>>> list(encoded_string)
[80, 89, 84, 72, 79, 78]
Bingo. It looks like UTF-8 assigns the letter ”P”
to the integer 80
, ”Y”
to the integer 89
, ”T”
to the integer 84
, and so on.
Let’s see what happens when we encode the string ”🇺🇸”
using UTF-8:
>>> list("🇺🇸".encode())
[240, 159, 135, 186, 240, 159, 135, 184]
Huh. Did you expect ”🇺🇸”
to get encoded as eight integers?
”🇺🇸”
is made up of two characters, namely “🇺”
and ”🇸"
. Let's see how those get encoded:
>>> list("🇺".encode())
[240, 159, 135, 186]
>>> list("🇸".encode())
[240, 159, 135, 184]
OK, things are making more sense now. Both “🇺”
and ”🇸"
get encoded as four integers, and the four integers corresponding to “🇺”
appear first in the list of integers corresponding to ”🇺🇸”
, while the four integers corresponding to ”🇸"
appear second.
This raises a question, though.
The character “🇺”
is encoded as a sequence of four integers in UTF-8, while the character ”P”
gets encoded as a single integer. Why is that?
There’s a hint at the top of Wikipedia’s UTF-8 article:
"UTF-8 is capable of encoding all 1,112,064 valid character code points in Unicode using one to four one-byte (8-bit) code units. Code points with lower numerical values, which tend to occur more frequently, are encoded using fewer bytes."
— Wikipedia, "UTF-8"
OK, so that makes it sound like UTF-8 isn’t encoding characters to integers, but instead to something called a Unicode code point. And each code unit can apparently be one to four bytes.
There are a couple of questions we need to answer now:
The word byte has been floating around a lot, so let’s go ahead and give it a proper definition.
A bit is the smallest unit of information. A bit has two states, on or off, that are usually represented by the integers 0
and 1
, respectively. A byte is a sequence of eight bits.
You can interpret bytes as integers by viewing their component bits as expressing a number in binary notation.
Binary notation can look pretty exotic the first time you see it. It's a lot like the usual decimal representation you use to write numbers, though. The difference is that each digit can only be a 0 or a 1, and the value of each place in the number is a power of 2, not a power of 10:
Since a byte contains eight bits, the largest number you can represent with a single byte is 11111111
in binary or 255
in decimal notation.
A character encoding that uses one byte for each character can encode a maximum of 256 characters since the maximum 8-bit integer is 255
and possible values start at 0
.
256 characters might be enough to encode everything in the English language. Still, there’s no way that it can handle all of the characters and symbols used in written and electronic communication worldwide.
So what do you do? Allowing characters to be encoded as multiple bytes seems like a reasonable solution, and that’s exactly what UTF-8 does.
UTF-8 is an acronym for Unicode Transformation Format — 8-bit. There's that word Unicode again.
According to the Unicode website:
"Unicode provides a unique number for every character, no matter what the platform, no matter what the program, no matter what the language."
— Unicode website, "What is Unicode?"
Unicode is massive. The goal of Unicode is to provide a universal representation for all written language. Every character gets assigned to a code point — a fancy word for “integer" with some additional organization — and there are a total of 1,112,064 possible code points.
How Unicode code points actually get encoded depends, though. UTF-8 is just one character encoding implementing the Unicode standard. It divides code points into groups of one to four 8-bit integers.
There are other encodings for Unicode. UTF-16 divides Unicode code points into one or two 16-bit numbers and is the default encoding used by Microsoft Windows. UTF-32 can encode every Unicode code point as a single 32-bit integer.
But wait, UTF-8 encodes symbols as code points using one to four bytes. OK, so… why does the 🇺🇸 symbol get encoded with eight bytes?
>>> list("🇺🇸".encode())
[240, 159, 135, 186, 240, 159, 135, 184]
>>> # There are eight integers in the list, a total of eight bytes!
Remember, two characters make up the US flag emoji: 🇺 and 🇸. These characters are called regional indicator symbols. There are twenty-six regional indicators in the Unicode standard representing A–Z English letters. They’re used to encode ISO 3166-1 two-letter country codes.
Here’s what Wikipedia has to say about regional indicator symbols:
"These were defined in October 2010 as part of the Unicode 6.0 support for emoji, as an alternative to encoding separate characters for each country flag. Although they can be displayed as Roman letters, it is intended that implementations may choose to display them in other ways, such as by using national flags. The Unicode FAQ indicates that this mechanism should be used and that symbols for national flags will not be directly encoded."
— Wikipedia, "Regional indicator symbol"
In other words, the 🇺🇸 symbol — indeed, the symbol for any country's flag — is not directly supported by Unicode. Operating systems, web browsers, and other places where digital text is used, may choose to render pairs of regional indicators as flags.
Let’s take stock of what we know so far:
So, when you reverse a string, what gets reversed? Do you reverse the entire sequence of integers in the encoding, or do you reverse the order of the code points, or something different?
Can you think of a way to answer this question with a code experiment rather than trying to look up the answer?
You saw earlier that UTF-8 encodes the string ”PYTHON”
as a sequence of six integers:
>>> list("PYTHON".encode())
[80, 89, 84, 72, 79, 78]
What happens if you encode the reversal of the string ”PYTHON”
?
>>> list("PYTHON"[::-1].encode())
[78, 79, 72, 84, 89, 80]
In this case, the order of the integers in the list was reversed. But what about other symbols?
Earlier, you saw that the “🇺"
symbol is encoded as a sequence of four integers. What happens when you encode its reversal?
>>> list("🇺".encode())
[240, 159, 135, 186]
>>> list("🇺"[::-1].encode())
[240, 159, 135, 186]
Huh. The order of the integers in both lists is the same!
Let’s try reversing the string with the US flag:
>>> list("🇺🇸".encode())
[240, 159, 135, 186, 240, 159, 135, 184]
>>> # ^^^^^^^^^^^^^^---Code point for 🇺
>>> # ^^^^^^^^^^^^^^^^^^---Code point for 🇸
>>> # The code points get swapped!
>>> list("🇺🇸"[::-1].encode())
[240, 159, 135, 184, 240, 159, 135, 186]
>>> # ^^^^^^^^^^^^^^---Code point for 🇸
>>> # ^^^^^^^^^^^^^^^^^^---Code point for 🇺
The order of the integers isn’t reversed! Instead, the groups of four integers representing the Unicode code points for 🇺and 🇸get swapped. The orders of the integers in each code point stay the same.
The title of this article is a lie! You can reverse a string with a flag emoji. But reversing symbols composed of multiple code points can have surprising results. Especially if you've never heard of things like character encodings and code points before.
But, is reversing the order of the code points, like Python does, the right approach? Would it make more sense to keep symbols represented by multiple code points intact? The answer is: it depends. There isn't a canonical way to reverse a string, at least that I'm aware of.
There are a couple of important lessons to take away from this investigation.
First, if you don't know which character encoding was used to encode some text, you can't guarantee that the decoded text accurately represents the original text.
Second, although UTF-8 is widely adopted, there are still many systems that use different character encodings. Keep this in mind when reading text from a file, especially when shared from a different operating system or across international borders. Be explicit and always indicate which encoding is being used to encode or decode text.
For example, Python’s open()
function has an encoding
parameter that specifies the character encoding to use when reading or writing text to a file. Make use of it.
We’ve covered a lot of ground, but there are still a lot of questions left unanswered. So write down some of the questions you still have and use the investigative techniques you saw in this article to try and answer them.
Here are some questions you might want to explore:
"🏴"
to a list, you end up with a bunch of strings that start with "\U"
. What are those strings, and what do they represent?"🏴"
contains a whopping 28 bytes of information. What makes 🏴 different from 🇺🇸? What other flags get encoded as 28 bytes?Thanks for reading! Stay curious out there!
It's easy to be complacent about curiosity. Our lives are filled with stress. We craft routines around that stress. Sometimes those routines turn life into an inescapable turnstile. It isn't easy to be curious when your life depends on keeping the wheel turning.
I spend a lot of my time solving problems with Python. Problem-solving demands creativity. Curiosity breeds creativity. The demand for creative software solutions is constant, but curiosity comes and goes.
When your curiosity wanes, don't resign yourself to the idea that some people are naturally more curious about things and that perhaps you're not one of them. Curiosity is not an immutable trait. You can learn how to be more curious.
Curiosity is contagious. Follow curious people, and you'll likely catch the bug. Once your mind is infected with a framework for curiosity, you instinctively apply that framework to everything.
Mystery is the key to curiosity. Embrace feeling puzzled because puzzles are everywhere. Question absolutes and axioms, even the ones in this article. Devise experiments to test premises.
Learn to recognize when your curiosity ebbs. I know that my curiosity is critically low whenever I stop reading consistently. I'm not talking about reading code — I mean reading anything. It's a case of intellectual malnutrition, and the cure is a content-rich diet.
Get into a habit of passive consumption. Listen to podcasts during your commute or household chores. Watch documentaries during your lunch break. If your mind wanders, let it. Don't focus on one topic. A diverse content diet is more likely to reveal a mystery that stirs you.
Actively consume content about mysteries that particularly draw your attention. Read long-form articles and books. Listen intently and take notes. But refrain from forcing yourself into active consumption. Forced activity stifles joy.
Unlike diets that nourish our bodies, a curiosity-inducing content diet has no plan and no schedule. Explore topics haphazardly, but also explore them deeply. Don't delay your curiosity. Pursue topics that interest you right now.
Get lost in rabbit holes, but avoid ones generated algorithmically. Algorithms have a knack for surfacing similar content. Actively search for answers to questions that arise.
Curiosity is a tug-of-war between selfishness and humility. You're compelled by a primal urge to know more. Yet learning requires you to expose a gap in your understanding, to recognize that there is still more to be known.
Learning is a conversation. Participate in that conversation. Ask questions. A good question synthesizes existing knowledge and addresses an acknowledged gap in understanding.
Getting better at being curious requires practicing curiosity. Fortunately, coding is full of puzzles. There are packages and tools to discover, languages to learn, and implementation details to explore.
Here are two techniques I've used to practice curiosity while coding.
Push boundaries and perform experiments.
I once worked as a programmer for a commercial audio/visual installation company. On one job, we installed a distributed video switching system controlled by an app. The app sent TCP packets from an iPad to a network switch and used VLAN untagging to connect video receivers on the network to broadcasts streamed by transmitters.
The app was incompatible with the latest switch firmware. The IT department refused to downgrade the firmware, and the app developer couldn't deliver a patch until well after the project deadline. But I could configure the app to send commands to any IP address.
I could write a TCP server to accept commands from the app, convert them to the network switch's protocol, and forward them to the switch. I needed a machine to host the script. The audio processing unit (APU) was a Linux box with plenty of storage and memory. Installing a custom script would void the warranty.
The APU was programmed by a drag-and-drop visual coding interface. One of the available "blocks" could run Lua scripts inside the application's sandbox. But, could a script running in the sandbox receive commands from the app and communicate with the network switch?
There were no documented restrictions, so I ran an experiment. It worked! Even better, script blocks automatically started whenever the APU booted. Video switching worked effortlessly even after reboots from power outages and other incidents.
My curiosity paid off. We completed the job on time. My discoveries created new solutions for clients and spawned internal tools that saved time and money on installations.
Destruction can be productive.
I enjoy finding ways to "break" a language. It isn't about finding bugs. It's about finding code examples that exhibit surprising behavior. It always starts with a question.
"How does a Python set determine if two objects are distinct?"
Knowing where to start looking for answers is a crucial skill. The Python docs on sets are pretty good.
"Oh, ok. Sets use an algorithm that checks if two objects have the same hash value and compare equal to each other. If both checks are true, then those objects are indistinguishable from each other, and the set can only contain one of them."
A little bit of knowledge opens the door to new questions.
"Can you 'trick' a set into thinking two different objects are nondistinct?"
Now there's a fun little puzzle. Think about it. An integer hashes to its value. You can control a custom object's hash value with the .__hash__()
method. That means you can create an object that hashes to the same value as the integer 1:
class NotOne:
def __hash__(self):
return 1
To confuse a set into thinking NotOne
instances are equal to 1
, you need them to compare equal to 1
. You can make NotOne
objects compare equal to any object by implementing an .__eq__()
method that always returns True
:
class NotOne:
def __hash__(self):
return 1
def __eq__(self, other):
return True
Now see what happens when you create a set containing 1
and an instance of NotOne
:
>>> n = NotOne()
>>> # Create a set S containing 1 and n
>>> S = {1, n}
>>> # S only contains 1
>>> S
{1}
>>> # But somehow n is in S!
>>> n in S
True
n
is very much a distinct object from 1
. n
isn't even a number. It doesn't support arithmetic operations. But you'll never be able to put n
and 1
in a set together because they fail to meet a set's criteria for distinctness. It feels weird that any set containing 1
also contains n
.
"That's pretty weird. I wonder if there's a way to trick a set into thinking two nondistinct objects are distinct from each other?"
For such a thing to be possible requires an object that doesn't compare equal to itself. If you've ever worked with IEEE 754 NaN objects before, you know they fit the bill.
"What happens when you try to put several NaN values into a Python set?"
Let's find out.
>>> S = {float("nan"), float("nan"), float("nan")}
>>> S
{nan, nan, nan}
"Ok, that is weird. But surely you can verify that the set contains a NaN object. Right?"
Behold:
>>> float("nan") in S
False
I love examples like this. In all honesty, nothing is broken — except maybe my brain for a little bit. Seeking out and understanding examples like these strengthens your intuition about code and surfaces new ideas for solving problems.
Start practicing curiosity today. Ask more questions and do more experiments. Be thankful that it's impossible to know everything. There's always a new puzzle to be solved. There are always new things to wonder about.
Every week, I share curiosity-inducing code snippets, ideas, and resources in my Curious About Code newsletter. Sign up to get a little curiosity boost in your inbox every Friday.
Here are five resources to help you learn more about the science of curiosity and cultivate more of it for yourself:
People complain that the iPad Pro isn't "pro" enough. I mean, Apple released a tablet with a leading processor and up to 16GB of RAM. With no support for pro apps like coding IDEs!
Or did they? 🤔
It's true: you can't install Visual Studio Code on your iPad, natively. But there are several professional-grade coding apps in the App Store, especially for Python programmers… if you know where to look.
Here are five ways you can code in Python on any iPad right now.
My goto app for using Python on the iPad is Nicolas Holzschuch’s fantastic a-Shell app. a-Shell gives you a Unix-style terminal on your iPad or iPhone and it’s completely free. It’s also quite powerful.
Click here to open a-Shell in the Apple Store \(\rightarrow\)
Once you install and open a-Shell, type help
to get a quick overview of the app:
a-Shell comes with vim
and ed
for editing files, and it includes Python 3.9 out-of-the box. Here’s what editing a Python file in vim
looks like:
Very nice!
Esc
key. This makes working in vim painful until you figure out that Cmd + .
works like Esc
. If you'd like, you can change Caps Lock
to work as Esc
in a-Shell’s settings.Esc
in the Settings app.a-Shell plays nicely with iPadOS 15’s multi-tasking features. You can open new windows, put a-Shell side-by-side with another app, and — my favorite — use a-Shell in slideover mode.
I do a lot of reading on my iPad. When I come across something that I want to check in the Python REPL, it’s incredibly helpful to be able to swipe in from the right-hand-side of my iPad, quickly check something in the REPL, and then dismiss the app by swiping right:
You can install Python packages using pip
in a-Shell as long as those package are pure Python. This is admittedly a serious limitation for a lot of folks, but it does allow you to install some pretty awesome packages — including Will McGugan’s awesome rich library:
Besides being a great way to use Python on your iPad, a-Shell has a lot of other useful features. You can navigate your iPad’s file system, transfer files using curl
, generate SSH keys, SSH into remote servers, and more. You can even write programs in C and C++ and run them on your iPad 🤯
One of a-Shell’s major downsides is the lack of support for Python virtual environments. This means a-Shell is great for testing things out, or for doing some basic, pure-Python programming, but it’s not very well suited to professional development.
One email, every Saturday, with one actionable tip.
Always less than 5 minutes of your time.
Carnets is a free, standalone Jupyter notebook app available on iPad and iPhone. You get a full Python and Jupyter implementation — no need to connect to an external Jupyter server — as well as a handful of useful Python packages, including NumPy, pandas, and Matplotlib.
Click here to view Carnets on the App Store \(\rightarrow\)
You can create, view, and edit notebook files, including ones that you created elsewhere or were sent to you by a colleague. The thing that I like most about it is that it “just works.” Download the app and in a few minutes you’re running Jupyter notebooks right on your iPad.
Carnet’s interface looks just like Jupyter in a browser. But what you see is what you get. There aren’t any bells and whistles here.
If you need to install a package that doesn’t come with Carnets, you can use %pip install
inside of a notebook cell to install the package:
To see all of the Python packages you get with Carnets, run %pip list
. There are quite a few — although many you see in the following list were installed by me or as dependencies of packages I installed:
There are two versions of Carnets available in the App Store:
1. Carnets – Jupyter
2. Carnets – Jupyter (with scipy)
Carnets – Jupyter (with scipy) includes a few additional packages for doing machine learning right on your iPad: scipy
, seaborn
, sklearn
, and coremltools
. If you can afford the extra space, I highly recommend downloading Carnets – Jupyter (with scipy) instead of the base Carnets app.
Like a-Shell, the Carnets app doesn’t let you create isolated Python environments. You’re stuck using the global environment and whatever package versions come pre-built with the app.
Juno is another Jupyter notebook app for the iPad that bills itself as a Jupyter IDE. Like Carnets, you get Python bundled with some extra packages. Unlike Carnets, Juno costs $15 and comes with some nice bonus features.
Click here view Juno on the App Store \(\rightarrow\)
Juno really stands out from Carnets with its sleek and modern iPad interface:
You can run cells by pressing Shift + Enter
inside of the cell or by tapping the blue Run Cell button at the bottom right-hand corner of the screen.
The lightning bolt button gives you quick access to some common tasks. You can change the cell type from code to Markdown, move cells up and down, and cut, copy, paste, and delete cells:
While Carnets can open Jupyter notebooks from anywhere on your iPad, the interface isn’t very iPad-friendly. Contrast this to Juno’s file picker, which really feels at home on iPad and iOS devices:
Another area where Juno shines is the IDE-like tab completion and tooltips that come built-in:
Like Carnets, Juno comes with a suite of built-in packages. But you can’t run %pip list
in a cell to see them all like you can in Carnets:
Package management in Juno is actually a bit of a disappointment, especially for an app that costs $15.
I couldn’t find a complete list of packages that come pre-installed with Juno, but here are a few imports that worked out of the box:
To install a package, click the name of the notebook at the top-center of the screen and select Install Python Package. You’ll see the following dialog box:
Like Carnets and a-Shell, you can only install pure Python packages. But unlike Carnets and a-Shell, any dependencies of a package you install won’t be installed automatically. You’ll need to install them manually one-by-one.
One nice thing about Juno’s package manger is that you get a chance to see some metadata for a package before you install it, including dependencies — which you can install at the touch of a button:
As I mentioned before, you can’t use %pip list
in Juno to view the packages you have installed into Juno’s environment. In fact, there is no way to view your installed packages from within Juno.
Instead, you must open the Files app and navigate to the site_packages/
folder in Juno’s on-device storage:
If you want to remove one of your installed packages, you need to do that manually from within site_packages/
. I find this to be a major shortcoming. I really appreciate the quick access to pip
using the %pip
magic command supported by Carnets.
Despite the clunky package manager, Juno does look much nicer than Carnets and the tab-completion and tooltips do boost productivity. If those features matter to you and you’re willing to fork over the $15, then Juno is a nice option.
Juno Connect is a Jupyter notebook client app that can be used to access Jupyter on a remote server. So, technicaly, Python isn’t running on your iPad, but Juno Connect provides a beautiful interface for working with remote notebook servers. You can purchase it from the App Store for $10.
Click here to view Juno Connect on the App Store \(\rightarrow\)
When you first launch Juno Connect, you’ll a see a screen with some notebooks ready to try out, as well as several options to connect to remote notebook servers:
Juno Connect support services like Cocalc and Binder right out of the box. You can also connect to Jupyter servers hosted elsewhere, such as on a Digital Ocean droplet.
When you connect to a notebook server, you’ll see a screen like the one below, where you can create a new notebook or select one to open:
Once you open or create a new notebook, you’ll see an interface that looks exactly like the typical Juno interface. And since the notebook is running on an external server, you get access to all of the typical Jupyter features, including the %pip
magic that doesn’t work with the local-only version of Juno:
One nice feature of Juno Connect is the ability to export a remote notebook to a local file on your iPad:
You can even export the notebook as a new notebook, allowing you to save a local copy to work with offline in either Juno or the Carnets app.
The last option on my list isn’t an iPad app and it doesn’t run Python locally on your iPad, but it’s absolutely essential if you need access to a full-blown development environment. That solution is the Codeanywhere Cloud IDE.
Plans start at $6 per month and you can get a 40% discount if you prepay for two years. This unlocks a VS Code style IDE that runs entirely in your browser.
Click here to get a 10% discount off any Codeanywhere subscription \(\rightarrow\)
Once you choose a plan and create an account, you’ll be taken to your dashboard where you can create new containers for working on projects:
When you click on New Container, you’ll have the opportunity use a blank Ubuntu Linux container or to select from some pre-defined containers with various languages pre-installed:
Once you create a container, it takes a few minutes for it to spin-up and become available to you in your dashboard:
The resources you get per container depend on the plan you selected. My plan gives me 15GB of storage and 4GB of memory.
Click the green Open IDE button to launch the IDE in a new browser tab:
If you’re familiar with VS Code, you’ll feel right at home in Codeanywhere’s IDE. It’s incredibly full-featured, including things like linter and debugging support:
You can even install extensions!
Codeanywhere is pricy compared to the other options mentioned in this list. It also requires an internet connection. But, I can’t live without it. It’s the best way I’ve found to do some hardcore coding on my iPad.
The five tools I've mentioned are what I currently use day-in and day-out to run Python on my iPad, but there are a number of other options that deserve a mention.
Whenever I bring up coding in Python on the iPad, I get a flurry of comments praising the Pythonista3 app. There was a time when Pythonista3 was a great option, but that’s no longer the case in my opinion.
First of all, Pythonista3 appears to be completely abandoned. The last version was released over a year ago before iPadOS 14 was available (the current version of iPadOS at the time of writing is 15). And, second, Pythonista3 only supports Python 3.6, which reached it’s end of life in December 2021.
The PyTo app is a serious contender in the Python-for-iPad space. You get Python 3.10 running locally with a nice IDE-style experience, support for editing multiple files, several bonus packages including NumPy, SciPy, Scikit-learn, and statsmodels. The full app experience costs $10.
I'm currently giving PyTo a run for it's money to see how it fits into my daily use. If it lives up to it's promise, you might find it higher up on the list in a future update to this article!
Google’s Colab is a browser-based notebook based on Jupyter notebooks. Your code runs on a private virtual machine that comes with most of the Python scientific stack pre-installed. You even get free access to GPUs, which makes Colab a great option for machine learning projects. If you need guaranteed uptime and more power, check out Colab Pro.
If you don't need Python running locally but want a solid IDE experience, check out vscode.dev. You can quickly clone an external code repository and start coding with Python right from your browser. The Python language doesn't have first-class support yet — stick with "webby" languages like JavaScript and CSS for the best experience — but it could be a good, free alternative in a pinch.
I've never used GitHub's Codespaces project, but if you have a GitHub organization on the Team or Enterprise plan, this might be a good option. It looks very similar to Codeanywhere.
The pyodide project brings Python and over seventy-five package in Python's scientific stack to any browser by compiling them all to WebAssembly. You can try pyodide in a REPL and bookmark it for quick access in the future.
Another option is to connect your iPad to a Raspberry Pi via USB, which allows you to access the Raspberry Pi as an ethernet device. This is really cool, but personaly I don’t want to have to carry any more devices around with me than I need to. It does make for a fun little project, though, and gets you access to a full local development server.
I’d love to see a native version of an IDE like VS Code on the iPad, but I don’t think it’s ever going to happen. However, between a-Shell, Carnets, and Codeanywhere, I’ve been able to do everything that I’ve needed to from my iPad. Of course, your mileage may vary.
So, in 2022, coding in Python on the iPad is not only feasible, it’s actually quite fun!
If you've done any coding in Python, there's a good chance that you've used a number in one of your programs. For instance, you may have used an integer to specify the index of a value in a list.
But there's much more to numbers in Python than just their raw values. Let's look at three things about numbers in Python that you might not be aware of.
Pretty much everything in Python is an object. One of the first objects you learn about in Python is the str
object for representing strings. Maybe you've seen how strings have methods, such as the .lower()
method, which returns a new string with all lower case characters:
>>> "HELLO".lower()
'hello'
Numbers in Python are also objects and, just like strings, have their own methods. For instance, you can convert an integer to a byte string using the .to_bytes()
method:
>>> n = 255
>>> n.to_bytes(length=2, byteorder="big")
b'\x00\xff'
The length
parameter specifies the number of bytes to use in the byte string, and the byteorder
parameter determines the order of the bytes. For example, setting byteorder
to " big"
returns a byte string with the most significant byte first, and setting byteorder
to " little"
puts the least significant byte first.
255 is the largest integer that can be represented as an 8-bit integer, so you can set length=1
in .to_bytes()
with no problem:
>>> n.to_bytes(length=1, byteorder="big")
b'\xff'
If you set length=1
in .to_bytes()
for 256, however, you get an OverflowError
:
>>> n = 256
>>> n.to_bytes(length=1, byteorder="big")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
OverflowError: int too big to convert
You can convert a byte string to an integer using the .from_bytes()
class method:
>>> int.from_bytes(b'\x06\xc1', byteorder="big")
1729
Class methods are called from a class name instead of a class instance, which is why the .from_bytes()
method is called on int
above.
Floating point numbers also have methods. Perhaps the most useful method for floats is .is_integer()
which is used to check whether or not a float has no fractional part:
>>> n = 2.0
>>> n.is_integer()
True
>>> n = 3.14
>>> n.is_integer()
False
One fun float method is the .as_integer_ratio()
method, which returns a tuple containing the numerator and denominator of the fraction representing the floating point value:
>>> n = 0.5
>>> n.as_integer_ratio()
(1, 2)
Thanks to floating point representation error, though, this method can return some unexpected values:
>>> n = 0.1
>>> n.as_integer_ratio()
(3602879701896397, 36028797018963968)
If you need to, you can call number methods on number literals by surrounding the literals with parentheses:
>>> (255).to_bytes(length=1, byteorder="big")
b'\xff'
>>> (3.14).is_integer()
False
If you don't surround an integer literal with parentheses, you'll see a SyntaxError
when you call a method — although, strangely, you don't need the parentheses with floating-point literals:
>>> 255.to_bytes(length=1, byteorder="big")
File "<stdin>", line 1
255.to_bytes(length=1, byteorder="big")
^
SyntaxError: invalid syntax
>>> 3.14.is_integer()
False
You can find a complete list of methods available on Python's number types in the docs.
In mathematics, numbers have a natural hierarchy. For example, all natural numbers are integers, all integers are rational numbers, all rational numbers are real numbers, and all real numbers are complex numbers.
The same is true for numbers in Python. This “numeric tower” is expressed through abstract types contained in the numbers
module.
Every number in Python is an instance of the Number
class:
>>> from numbers import Number
>>> # Integers inherit from Number
>>> isinstance(1729, Number)
True
>>> # Floats inherit from Number
>>> isinstance(3.14, Number)
True
>>> # Complex numbers inherit from Number
>>> isinstance(1j, Number)
True
If you need to check if a value in Python is numeric, but you don't care what type of number the value is, use isinstance(value, Number)
.
Python comes with four additional abstract types whose hierarchy, beginning with the most general numeric type, is as follows:
Complex
class is used to represent complex numbers. There is one built-in concrete Complex
type: complex
.Real
class is used to represent real numbers. There is one built-in concrete Real
type: float
.Rational
class is used to represent rational numbers. There is one built-in concrete Rational
type: Fraction
.Integral
class is used to represent integers. There are two built-in concrete Integral
types: int
and bool
.Wait. bool
values are numbers?! Yes! You can verify all of this in your REPL:
>>> import numbers
>>> # Complex numbers inherit from Complex
>>> isinstance(1j, numbers.Complex)
True
>>> # Complex numbers are not Real
>>> isinstance(1j, numbers.Real)
False
>>> # Floats are Real
>>> isinstance(3.14, numbers.Real)
True
>>> # Floats are not Rational
>>> isinstance(3.14, numbers.Rational)
False
>>> # Fractions are Rational
>>> from fractions import Fraction
>>> isinstance(Fraction(1, 2), numbers.Rational)
True
>>> # Fractions are not Integral
>>> isinstance(Fraction(1, 2), numbers.Integral)
False
>>> # Ints are Integral
>>> isinstance(1729, numbers.Integral)
True
>>> # Bools are Integral
>>> isinstance(True, numbers.Integral)
True
>>> True == 1
True
>>> False == 0
True
At first glance, everything seems right here — other than bool
values being numbers, perhaps.
bool
type is Integral
— in fact, bool
inherits directly from int
— you can do some pretty weird stuff with True
and False
.True
as an index to the get the second value of an iterable. If you divide a number by False
you get a ZeroDivisionError
error."False"[True]
and 1 / False
in your REPL!Take a closer look, though, there are a couple of things that are a bit weird about Python's numeric hierarchy.
There are four concrete numeric types corresponding to the four abstract types in Pythons number tower: complex
, float
, Fraction
, and int
. But Python has a fifth number type, the Decimal
class, that is used to exactly represent decimal numbers and overcome limitations of floating-point arithmetic.
You might guess that Decimal
numbers are Real
, but you'd be wrong:
>>> from decimal import Decimal
>>> import numbers
>>> isinstance(Decimal("3.14159"), numbers.Real)
False
In fact, the only type that Decimal
numbers inherit from are Python’s Number
class:
>>> isinstance(Decimal("3.14159"), numbers.Complex)
False
>>> isinstance(Decimal("3.14159"), numbers.Rational)
False
>>> isinstance(Decimal("3.14159"), numbers.Integral)
False
>>> isinstance(Decimal("3.14159"), numbers.Number)
True
It makes sense that Decimal
doesn't inherit from Integral
. To some extent, it also makes sense that Decimal
doesn't inherit from Rational
. But why doesn't Decimal
inherit from Real
or Complex
?
The answer lies in the CPython source code:
Decimal has all of the methods specified by theReal
abc, but it should not be registered as aReal
because decimals do not interoperate with binary floats (i.e.Decimal('3.14') + 2.71828
is undefined). But, abstract reals are expected to interoperate (i.e. R1 + R2 should be expected to work if R1 and R2 are both Reals).
It all boils down to implementation.
On the one hand, floats implement the Real
abstract base class and are used to represent real numbers. But thanks to finite memory constraints, floating-point numbers are merely finite approximations of real numbers. This leads to confusing examples, like the following:
>>> 0.1 + 0.1 + 0.1 == 0.3
False
Floating-point numbers get stored in memory as binary fractions, but this causes some problems. Just like the fraction \(\frac{1}{3}\) has no finite decimal representation — there are infinitely many threes after the decimal point — the fraction \(\frac{1}{10}\) has no finite binary fraction representation.
In other words, you can't store 0.1 on a computer with exact precision — unless that computer has infinite memory.
From a strictly mathematical standpoint, all floating-point numbers are rational — except for float("inf")
and float("nan")
. But programmers use them to approximate real numbers and treat them, for the most part, as real numbers.
float("nan")
is a special floating point value representing "not a number" values — often abbreviated as NaN
values. But since float
is a numeric type, isinstance(float("nan"), Number)
returns True
.Floats are weird.
Python's abstract numeric base types allow you to create your own custom abstract and concrete numeric types.
As an example, consider the following class ExtendedInteger
which implements numbers of the form \(a + b\sqrt{p}\) where \(a\) and \(b\) are integers and \(p\) is prime (note that primality is not enforced by the class):
import math
import numbers
class ExtendedInteger(numbers.Real):
def __init__(self, a, b, p = 2) -> None:
self.a = a
self.b = b
self.p = p
self._val = a + (b * math.sqrt(p))
def __repr__(self):
return f"{self.__class__.__name__}({self.a}, {self.b}, {self.p})"
def __str__(self):
return f"{self.a} + {self.b}√{self.p}"
def __trunc__(self):
return int(self._val)
def __float__(self):
return float(self._val)
def __hash__(self):
return hash(float(self._val))
def __floor__(self):
return math.floor(self._val)
def __ceil__(self):
return math.ceil(self._val)
def __round__(self, ndigits=None):
return round(self._val, ndigits=ndigits)
def __abs__(self):
return abs(self._val)
def __floordiv__(self, other):
return self._val // other
def __rfloordiv__(self, other):
return other // self._val
def __truediv__(self, other):
return self._val / other
def __rtruediv__(self, other):
return other / self._val
def __mod__(self, other):
return self._val % other
def __rmod__(self, other):
return other % self._val
def __lt__(self, other):
return self._val < other
def __le__(self, other):
return self._val <= other
def __eq__(self, other):
return float(self) == float(other)
def __neg__(self):
return ExtendedInteger(-self.a, -self.b, self.p)
def __pos__(self):
return ExtendedInteger(+self.a, +self.b, self.p)
def __add__(self, other):
if isinstance(other, ExtendedInteger):
# If both instances have the same p value,
# return a new ExtendedInteger instance
if self.p == other.p:
new_a = self.a + other.a
new_b = self.b + other.b
return ExtendedInteger(new_a, new_b, self.p)
# Otherwise return a float
else:
return self._val + other._val
# If other is integral, add other to self's a value
elif isinstance(other, numbers.Integral):
new_a = self.a + other
return ExtendedInteger(new_a, self.b, self.p)
# If other is real, return a float
elif isinstance(other, numbers.Real):
return self._val + other._val
# If other is of unknown type, let other determine
# what to do
else:
return NotImplemented
def __radd__(self, other):
# Addition is commutative so defer to __add__
return self.__add__(other)
def __mul__(self, other):
if isinstance(other, ExtendedInteger):
# If both instances have the same p value,
# return a new ExtendedInteger instance
if self.p == other.p:
new_a = (self.a * other.a) + (self.b * other.b * self.p)
new_b = (self.a * other.b) + (self.b * other.a)
return ExtendedInteger(new_a, new_b, self.p)
# Otherwise, return a float
else:
return self._val * other._val
# If other is integral, multiply self's a and b by other
elif isinstance(other, numbers.Integral):
new_a = self.a * other
new_b = self.b * other
return ExtendedInteger(new_a, new_b, self.p)
# If other is real, return a float
elif isinstance(other, numbers.Real):
return self._val * other
# If other is of unknown type, let other determine
# what to do
else:
return NotImplemented
def __rmul__(self, other):
# Multiplication is commutative so defer to __mul__
return self.__mul__(other)
def __pow__(self, exponent):
return self._val ** exponent
def __rpow__(self, base):
return base ** self._val
You need to implement lots of dunder methods to ensure the concrete type implements the Real
interface. You also have to consider how methods like .__add__()
and .__mul__()
interact with other Real
types.
With ExtendedInteger
implemented, you can now do things like this:
>>> a = ExtendedInteger(1, 2)
>>> b = ExtendedInteger(2, 3)
>>> a
ExtendedInteger(1, 2, 2)
>>> # Check that a is a Number
>>> isinstance(a, numbers.Number)
True
>>> # Check that a is Real
>>> isinstance(a, numbers.Real)
True
>>> print(a)
1 + 2√2
>>> a * b
ExtendedInteger(14, 7, 2)
>>> print(a * b)
14 + 7√2
>>> float(a)
3.8284271247461903
Python's number hierarchy is quite flexible. But, of course, you should always take great care when implementing types derived from built-in abstract base types. You need to make sure they play well with others.
There are several tips in the docs for type implementors that you should read before implementing custom number types. It's also helpful to peruse the implementation of Fraction
.
So there you have it. Three things (plus a whole lot more, maybe) that you might not have known about numbers in Python:
Decimal
and float
.I hope you learned something new!
Disclosure: some of the links in this article are affiliate links.
Learning a new skill is exciting. But learning how to learn is a skill that we're never really taught. This is a real tragedy and one that hasn't gone unnoticed.
There are plenty of self-help gurus with courses designed to help you learn how to learn. But there's a lot of noise in this space, in my opinion. A lot of fluff. I want to cut through that noise.
There's really only one thing you need to know.
One of the biggest myths about learning is the so-called 10,000-hour rule. This "rule" states that you need to accumulate approximately 10,000 hours of deliberate practice to become an expert in something. The idea is based on research by Swedish psychologist K. Anders Ericsson, who found that top performers — think chess grandmasters and musical virtuosos — trained for at least 10,000 hours before winning their first international competition.
The 10,000-hour rule was popularized by author Malcolm Gladwell in his 2007 book Outliers: The Story of Success. Author and business coach Josh Kaufman observed that a "society-wide game of telephone" began with this book. What started as a statement about top performers morphed into a "rule" about learning in general. And it's patently untrue.
What the 10,000-hour rule gets right, though, is the importance of practice. But you don't need to practice for 10,000 hours to learn something new. Even K. Anders Ericsson observed that it takes a surprisingly short amount of time to become reasonably good at something:
Let's imagine you are learning to play golf for the first time. In the early phases, you try to understand the basic strokes and focus on avoiding gross mistakes (like driving the ball into another player). You practice on the putting green, hit balls at a driving range, and play rounds with others who are most likely novices like you. In a surprisingly short time (perhaps 50 hours), you will develop better control and your game will improve. (Source)
When you start to learn something new, you tend to improve exponentially. But, eventually, your improvement levels off, and even though you are still practicing, you aren't improving. This frustrating improvement plateau happens because you aren't practicing deliberately.
To keep improving, you need to develop the ability to self-edit. This requires repeating a task over and over again, recognizing small mistakes, and adjusting to correct them. Here's how K. Anders Ericsson, continuing the golf analogy, describes deliberate practice:
Your golf game now is a social outing, in which you occasionally concentrate on your shot. From this point on, additional time on the course will not substantially improve your performance, which may remain at the same level for decades.
Why does this happen? You don't improve because when you are playing a game, you get only a single chance to make a shot from any given location. You don't get to figure out how you can correct mistakes. If you were allowed to take five to ten shots from the exact same location on the course, you would get more feedback on your technique and start to adjust your playing style to improve your control. (Source)
I call this the repetition-refinement feedback loop, and I've experienced its power in my own life. Learn enough to recognize errors, repeat a task until mistakes become apparent, adjust performance to correct the mistakes. Rinse and repeat.
I firmly believe the repetition-refinement loop is the key to learning anything — and to continue to learn over a lifetime. So how do you do it? Throughout my years of learning, I've narrowed it down to five steps:
Let's pick apart each of the five steps in more depth.
No matter how large the task seems, every skill I've ever learned only required learning three to five things to get started. The trick is knowing what those subskills are. Do a bit of research and break the skill down into smaller subskills. Go as deep as you can and try to identify the essential skills you need to learn before starting your learning journey.
This isn't as easy as it sounds. Many skills are much easier to deconstruct with guidance from someone who knows more than you do. Find someone ahead of you on their learning path to help you identify the key things to focus on as a novice.
A great tool for deconstructing a skill is a mind map. A mind map is a visual way of organizing skills, topics, and ideas around a central concept. You only need some paper and a pencil to make one. If you like apps, I highly recommend MindNode.
Find two or three resources covering the subskills you identified in step one. These could be blog posts, books, online courses, notes from a class — anything that contains the fundamentals of the skill you want to learn.
The goal here is to learn just enough to self-edit. You need to be able to recognize when you're making a mistake. Don't get caught in tutorial hell. Learn a few things, then start practicing.
Take notes while you're learning the fundamentals. All you need is a pen and paper, but it's best if your notes are searchable so you can find what you're looking for in the future. If you have a smartphone, you might be able to take pictures of your notes and use text recognition to search them. Personally, I use the GoodNotes app on my iPad to take handwritten notes with the Apple pencil.
Test yourself often. Flashcards are a good way to create little quizzes that you can reuse. I use old-fashioned paper cards for this, but I know many people that prefer Anki.
Once you've learned a few fundamentals, it's time to get your hands dirty. Make some mistakes and break some things. It's all a normal part of the learning process!
The goal here is to become reasonably good at the skill. It's difficult to define what "reasonably good" means. In my experience, it boils down to being able to perform the skill without overthinking it. It does not mean performing the skill perfectly. Making mistakes at this point is perfectly OK.
Why fifteen hours? That's roughly 30 minutes a day for one month. Of course, the time it takes to be reasonably good at something will vary, but fifteen hours is usually enough time.
Don't cram your practice into long sessions. Instead, split it up across multiple days. Studies have shown that spaced practice leads to better long-term memory. On days that you aren't practicing one skill, practice another. Interleaving small, related skills improves your overall understanding by strengthening connections between the skills.
Keep a practice log and treat it like a journal. Record what you practiced, how you felt about it, what you did well and what you struggled with. I use Notion to log things like this, but good ol' paper and pencil work just as well.
A learning barrier is anything that keeps you from learning as efficiently as possible. Use your practice log to identify and track learning barriers. Make a note of anything that causes friction and experiment with ways to eliminate the problem.
There are four types of learning barriers:
Physical barriers include your location and your environment. Of course, the ideal location depends on the skill you're learning. For example, if you're training to become a competitive swimmer, you won't get far if you train in your bathtub.
Most of the skills I've learned involve a computer and require deep focus and concentration. I have difficulty concentrating with background noise, but a quiet location isn't always available to me. A good pair of noise-canceling headphones can transform a noisy environment into a calm, peaceful learning sanctuary.
Physical learning barriers are, in theory, the most manageable. Just change your location and environment. Finding what works best for you, though, requires experimentation.
Emotional barriers involve fears and anxieties you may have about learning. Learning exposes you to the unknown, and the unknown is scary. Everyone I know that took on learning something new was anxious about it, so if you experience anxiety, you're in good company.
Seek out a support group. They could be classmates or an online community. Despite its general tilt towards toxicity, even social media can provide a network of supporters.
Psychological barriers are things like ADD/ADHD, dyslexia, or autism. These are significant barriers to learning and the most difficult to control. The first step is to be aware of them. Talk to a medical professional if you believe you suffer from a psychological condition.
Physiological barriers, such as getting enough sleep and eating well, are, in my experience, easy to ignore. For example, if I don't set reminders for myself, I can easily skip a meal when caught up with a task.
Sleep is probably the most critical physiological barrier to control. When you sleep, your brain process everything you've learned that day and creates new connections. Studies have shown that exercise also plays a role in memory. I like to walk. Walking boosts my mood and improves my focus. It's also a great way to deal with frustration.
As soon as you know one thing about whatever you're learning, you know more than someone else who wants to learn but hasn't started. When you share this with someone else, you solidify your own knowledge. Teaching also exposes gaps in your understanding and helps you identify areas that need improvement.
Teaching is scary. I know because I've done it. A lot. When you teach, you are vulnerable. But you don't have to teach in-person in front of hundreds of people. You can teach via email or Twitter DMs. Join a community centered around whatever you're learning and help others.
I've shared my framework for learning. But a framework, however nice it looks on paper, is only worth its salt if it works in practice. So get to it, and let me know how it works for you!
I recently came across a tweet with a screenshot from a calculus book that defines the derivative with no mention of a limit. Not even Leibniz’s differentials have a home here. And although the definition is quite verbose, I found that it makes the derivative feel natural — tangible even.
If you’ve taken a calculus course in the last 120 years or so, then you’ve undoubtedly encountered the concept of a limit. In fact, a quick search for the question “Why are limits important?” returns hundreds, if not thousands, of results stating that limits are an essential part of calculus. How could you possibly understand the subject without them?
It turns out that you can develop a great deal of calculus — at least the entire first-year curriculum — without limits. And doing so, in my opinion, provides a deeper understanding of the subject, its history, and a better appreciation for limits and why they are useful.
While statements professing that you can’t do calculus without limits surely have Leibniz turning in his grave, it’s not surprising that students come away with this mindset. Limits are everywhere in the calculus curriculum.
Limits are abstract, though, and students who struggle to understand them will likely struggle with all of calculus. Some students will be doomed to failure, which is a tragedy considering Newton had little more than an intuition about limits. It took nearly a century after Newton for Cauchy and Weierstrass to formalize limits into the ϵ-δ definition used today.
Definition. Let f(x) be a function defined on an open interval around a (note that f(a) need not be defined). Then we say that the limit of f(x) as x approaches a is L and write
$$ \lim_{x \to a} f(x) = L $$
if for every number ϵ > 0 there exists some number δ > 0 such that
$$ |f(a) - L| < \epsilon \hspace{6pt} \textrm{whenever} \hspace{6pt} 0 < |x - a| < \delta $$
What makes this definition difficult to understand? In my opinion, the reasons are twofold:
Math educators, aware of students’ difficulties with limits, have endured their own struggle to invent better ways of teaching the topic. In 1981, Jerrold Marsden and Alan Weinstein, professors at the University of California, Berkely, proposed a new method — don’t teach limits.
Calculus without limits isn’t new. Gottfried Leibniz devised calculus using differentials, which are infinitesimal positive quantities less than any real number. The method of exhaustion, developed independently by the ancient Greeks and the Chinese, can be used to find areas and volumes of round shapes, like circles and cones.
Marsden and Weinstein took the latter approach and adapted the method of exhaustion to differentiation. In the preface to their book Calculus Unlimited, on the page screenshotted in the tweet that caught my attention, the authors present instructors with the idea of overtaking [1].
Definition. Let f and g be real-valued functions with domains contained in ℝ, and let a be a real number. We say that f overtakes g at a if there is an open interval I containing a such that
There’s a lot of notation in that definition, but the idea lends itself to a natural geometric interpretation.
For example, draw the graphs of two functions f and g that intersect at some point whose x-coordinate is a. If, around some interval I on the x-axis, the graph of f is below the graph of g to the left of a above the graph of g to the right of a, then f overtakes g at a.
To define the derivative, Marsden and Weinstein look at the slopes of lines through a.
Definition: Let f be a function defined on an open interval containing a. Consider the family of lines lₘ(x) = f(a) + m(x-a). Suppose there is a number b such that
Then we say that f is differentiable at a, and that b is the derivative of f at a.
Like the definition of overtaking, Marsden and Weinstein’s definition of the derivative paints a convenient geometric picture. It even provides a tangible method to find the derivative of a function at some point using graphing software.
For instance, consider the graph of f(x) = x² and some point, say a = 1. Graph any line that passes through f(a) = 1² = 1. Now vary the slope of the line until you find a new line that neither overtakes f nor is overtaken by f. If you can find such a line, its slope is the derivative of f(x) at a.
If you play around with this method in graphing software long enough, you’ll notice that you need to zoom in very close to the point f(a) to get any semblance of precision. I found this to be a beautiful and natural manifestation of what the ϵ-δ definition of a limit represents — namely the concern with points on f arbitrarily close to the limit — but manages to obscure from students through its cryptic formulation.
This visual method for finding the derivative at a point isn’t rigorous. It also lacks precision for all but the nicest of functions at the nicest of points. It does emphasize something, however, that was lost on me as a student for a long time: the tangent line is special.
In chapter 2 of Marsden and Weinstein’s textbook, the authors introduce the notion of a transition point as a point at which something suddenly changes. There are countless examples of transition points in nature: sunrise is the transition point from day to night, 100 degrees celsius is the point at which liquid water transitions to water vapor (at sea level, anyway).
The tangent line can be thought of as the transition point between the set of all lines through a point a that are overtaken by f and the set of all lines through a that overtake f. Something special happens with the tangent line that splits the set of all lines through the point a into two disjoint sets.
Transition points are perhaps the foundational concept in Calculus Unlimited. Marsden and Weinstein frame integrals through the lens of transition points, as well, and the concept plays an important role in their proof of The Fundamental Theorem of Calculus.
Marsden and Weinstein mention in their preface that “as far as [they] know, [their] definition [of the derivative] has not appeared elsewhere” [1]. That may be true of their specific definition. Still, a method established by Apollonius of Perga and later re-discovered by John Wallis calculates the derivative in a manner that feels very much like Marsden and Weinstein’s approach [2].
Math historian John Suzuki, in his essay The Lost Calculus, explains how the definition of the tangent line used by Apollonius and Wallis can be described in modern terms:
Suppose we wish to find the tangent to a curve y = f(x) at the point (a, f(a)). The tangent liney = T(x) may be defined as the line resting on one side of the curve; hence weither f(x) > T(x) for all x ≠ a, or f(x) < T(x) for all x ≠ a. (This is generally true for curves that do not change concavity; if the curve does change concavity, we must restrict our attention to intervals where the concavity is either positive or negative.)
Suzuki explores the work on derivatives by René Descartes, Jan Hudde, and Isaac Barrow, all of whom approached the problem without the need for infinitesimals and limits. Barrow even proved a version of The Fundamental Theorem of Calculus [2]!
There are many things that I like about Marsden and Weinstein’s textbook. But I’m not a calculus student anymore. I’ve been out of the classroom for nearly a decade, so it’s hard for me to say how students would respond to these ideas.
Marsden and Weinstein’s definition of the derivative is verbose, and it doesn’t address the quantifier complexity introduced by the formal definition of a limit. However, it provides a geometric algorithm for finding the derivative and avoids introducing logic that feels circular.
Although their approach has its appeal, a glance through the book is enough to see how arduous the calculations are using the book’s definitions. Whatever is gained in intuition is offset by painfully boring algebraic manipulations. Not to mention that removing limits from calculus is a disservice to students who aspire to become mathematicians.
There is a lesson here, though, that shouldn’t be overlooked.
Newton’s and Leibniz’s ideas are indisputably important, but framing calculus as the study of limits gives students an inaccurate picture of what calculus really is. So much emphasis is placed on the limit that, at least in many students’ minds, limits become synonymous with calculus itself. That’s like saying that painting is the art of using a paintbrush!
Even worse, focusing on limits hides centuries of effort that provides historical context and validates students who struggle with the modern definitions of the limit and derivative.
Exploring derivatives without limits provides the opportunity to better understand the content of calculus, not just the tools, as well as the long history behind the problems calculus solves. In doing so, you might just come away with a better appreciation of why limits are useful. I certainly did.
If you find an error in this article, you can report it here.
[1] Marsden, Jerrold and Weinstein, Alan J. (1981) Calculus Unlimited. Benjamin/Cummings Publishing Company, Inc. , Menlo Park, CA. ISBN 0–8053–6932–5. https://resolver.caltech.edu/CaltechBOOK:1981.001
[2] Suzuki, Jeff. (2005). The Lost Calculus. Mathematics Magazine, vol. 78, (2005), pp. 339–353. https://www.maa.org/programs/maa-awards/writing-awards/the-lost-calculus-1637-1670-tangency-and-optimization-without-limits
The diagrams and figures in this article were created with Canva and Geogebra.
Some numbers are hard to compute.
In a 1990 article for Scientific American, mathematicians Ronald Graham and Joel Spencer quoted Paul Erdös as saying:
Suppose aliens invade the earth and threaten to obliterate it in a year's time unless human beings can find the Ramsey number for red five and blue five. We could marshal the world's best minds and fastest computers, and within a year we could probably calculate the value. If the aliens demanded the Ramsey number for red six and blue six, however, we would have no choice but to launch a preemptive attack.
It's an intriguing quote. But what the heck is a Ramsey number, and why is it so hard to calculate?
Let's start by playing a game.
The game is called Sim — a two-player game that you can play with pencil and paper. Every game of Sim is played on a "board" made from the six vertices of a hexagon:
Each player takes turns drawing an edge between a pair of vertices, but they do so with two different colors. For example, player one might use a blue pencil, and player two might use a red pencil.
The first player to join any two of their existing edges into a triangle immediately loses:
But will one player always lose? Or is it possible for the game to end in a draw? Play a few rounds of the game with a friend — or against a computer — and see if you can force a tie!
To determine whether or not Sim can end in a draw, let's think about a different question — one that, at first glance, might not seem related at all.
Your friend messages you and says, "I want to host a party for a group of randomly selected people, but I need to make sure that either: 1) there are three people who are all mutually strangers, or 2) there are three people who are all friends with each other. I'm serving dinner, and I'm on a tight budget. What's the smallest number of people I need to invite?"
Let's see if we can help your friend out.
Since we need to guarantee at least three mutual strangers or three people who are all friends, it seems reasonable that we need to invite at least three people to the party.
But three people isn't enough to guarantee your friend's conditions are met. For example, your friend might invite two people that know each other and one that is a stranger to the other two:
Four people won't work, either. Your friend could invite two people who are friends and two people who are strangers to each other but friends with one other person at the party:
What about five people? That feels more promising because even if your friend invites two people who are friends, the remaining three people could all be mutual strangers:
But the situation isn't so simple. There are lots of ways that five people could be in stranger/friend relationships with one another. For instance, you might find a group of three people out of the five where Person 1 knows Person 2 and Person 2 knows Person 3, but Person 1 doesn't know Person 3:
You can extend this situation to five people so that everyone at the party is friends with exactly two other people and a stranger to everyone else.
How? Well, let's represent each of the five people at the party as the vertex of a pentagon and use blue edges to indicate two people are friends and red edges to indicate two people are strangers.
Join each of the vertices around the face of the pentagon using blue edges. Then connect each pair of vertices that aren't joined by a blue edge with a red edge:
The diagram must have three vertices joined into a triangle with all blue edges to have three mutual friends. The same goes for three mutual strangers, except the edges of the triangle need to be red.
But there aren't any triangles in the diagram with three edges of the same color! So inviting five people to the party won't guarantee that your friend's conditions are met.
At this point, you might be wondering if there is any number of people your friend can invite to the party so that at least three people are mutual friends or three people are mutual strangers.
Does a similar diagram rule out six people? For example, what happens if you draw the vertices of a hexagon, join all of the vertices around the face with blue edges, and then connect everything else with red edges?
Well, look at that! Our first triangle! The strategy that worked for five people fails for six, but it doesn't mean that any group of six will necessarily contain three mutual strangers.
Do you notice any similarities between your friend's dinner party problem and the Game of Sim?
In both cases, we've drawn the vertices of a polygon and added blue and red edges. The difference is that in Sim, you try to avoid triangles with the same color edges, and in the dinner party problem, you want to form triangles whose edges are all the same color.
Although explaining the diagrams in terms of polygons helps us visualize the problem, it's not strictly necessary. What we've really been doing is drawing graphs — not the graphs of functions you might be familiar with from algebra, but a different kind of graph, comprised of vertices and edges. These graphs are studied in an area of mathematics called graph theory.
In particular, we've been drawing complete graphs, which are graphs with every possible edge between every pair of distinct vertices.
The Game of Sim and the dinner party problem are concerned with the inevitability of certain kinds of structures appearing whenever you color all the edges in a complete graph with two colors. These structures are called subgraphs.
A subgraph is a portion of a graph comprised of one or more of the vertices in the original graph and a subset of the edges in the original graph that connect the vertices in the subgraph.
If a subgraph is complete, it is called an \(n\)-clique, where n is the number of vertices in the subgraph.
Using the language of graph theory, you can reframe the dinner party problem as a question about graphs.
The Dinner Party Problem (Graph-Theoretical Version): Does a complete graph exist for which any red-blue coloring of its edges results in either a red 3-clique or a blue 3-clique? And if one does exist, what is the smallest one?
You can also reframe the question about whether or not the Game of Sim can end in a draw in graph-theoretical terms.
The Game of Sim (Graph-Theoretical Version): Does every red-blue coloring of the complete graph on six vertices contain either a red 3-clique or a blue 3-clique?
In 1928, British philosopher and mathematician Frank Ramsey published a paper called On a Problem in Formal Logic. The subject of Ramsey's paper bears little resemblance to the dinner party problem or the Game of Sim.
But one of the results in the paper — one that was not the focus of the discussion but needed in a more important argument — showed that for a sufficiently large system, no matter how chaotic, there exists a substructure with some kind of order. This minor result became one of Ramsey's most iconic legacies, and it can be stated in graph-theoretical terms.
Ramsey's Theorem (Graph-Theoretical Version): Given any two positive integers \(r\) and \(b\), there exists a minimum number \(R(r, b)\) such that any red-blue coloring of the complete graph on \(R(r, b)\) vertices contains either a red \(r\)-clique or a blue \(b\)-clique.
The number \(R(r, b)\) is called a Ramsey number, and the notation reminds us that Ramsey numbers depend on the choice of \(r\) and \(b\).
If you set both \(r\) and \(b\) to 3, then Ramsey's Theorem looks a lot like the dinner party problem. In fact, the theorem tells you that a solution to the problem must exist! And the smallest number of vertices you need — that is, the smallest number of people your friend needs to invite to the party — is \(R(3, 3)\).
Telling your friend to invite \(R(3, 3)\) people to their party isn't particularly helpful, though. It's good to know that they can solve their problem, but they need to know the value of \(R(3, 3)\).
The analysis we did for the dinner party problem tells us that \(R(3, 3)\) is at least six. But just because we found one red-blue coloring of the edges for the complete graph on six vertices with a red 3-clique doesn't mean that all red-blue edge colorings will have that property.
Let's look at the situation in a bit more depth. First, choose any vertex in the complete graph on six vertices and call it \(v\).
The vertex \(v\) has five edges connected to it. Now, pretend that you've colored all of the edges in the graph blue or red. What can you say about the number of red and blue edges connected to \(v\)?
There are six possibilities:
In all six cases, \(v\) is connected to at least three edges of the same color. For the sake of argument, assume that \(v\) is connected to at least three red edges. (If it isn't, swap red and blue in the following argument.) Let \(x\), \(y\), and \(z\) be the vertices at the other end of those edges.
In the diagram, there are three specific edges colored red, but we could have chosen any three of the five edges connected to \(v\).
If any of the edges \(xy\), \(xz\), or \(yz\) is red, then the coloring contains a red 3-clique.
On the other hand, if none of the edges \(xy\), \(xz\), or \(yz\) is red — in other words, they are all blue — then the coloring contains a blue 3-clique.
So, any red-blue edge coloring of the complete graph on six vertices inevitably contains either a red 3-cycle or a blue 3-cycle. This means that \(R(3, 3)\) is at most six, but since we already knew that \(R(3, 3)\) is at least six, we can conclude that \(R(3, 3)\) is exactly six!
This simultaneously solves the dinner party problem and whether the Game of Sim can end in a draw.
You can tell your friend that they only need to invite six people to the party to guarantee that at least three people are mutual friends or are mutual strangers.
And since the Game of Sim iteratively colors the edges of a complete graph on six vertices red or blue, at some point, one of the players will be forced to complete a triangle whose edges are the same color.
Interest in Ramsey's Theorem seems to have begun when the 1953 Putnam Competition included a graph-theoretic version of the dinner party problem. The solution to the problem established that \(R(3, 3)\) is less than six, and in 1955, mathematicians Robert Greenwood and Andrew Gleason proved that \(R(3, 3)\) is equal to six.
In the same paper, Greenwood and Gleason showed that \(R(4, 4)\) is eighteen using techniques considerably more advanced than what is needed for \(R(3, 3)\).
The search for \(R(5, 5)\), and Ramsey numbers for different values of \(r\) and \(b\), was on. In 1965, H. L. Abbott published the first lower bound for \(R(5, 5)\) in his doctoral thesis, setting the value to at least thirty-eight. By 1989, the value had been raised to forty-three.
The current best upper bound for \(R(5, 5)\) is forty-eight, and it took over a decade to lower that from the previous upper bound of forty-nine. So, after more than half a century, the best estimates for \(R(5, 5)\) are:
$$ 43 \leq R(5, 5) \leq 48. $$
The situation is even worse for \(R(6, 6)\). The current best estimates are:
$$ 102 \leq R(6, 6) \leq 165. $$
Why is calculating Ramsey numbers so hard, even for small values of \(r\) and \(b\)? It's all thanks to an explosion in the number of possible edge colorings as the number of vertices grows in the complete graphs that need to be checked.
Every vertex in a complete graph on \(n\) vertices is connected to \(n-1\) edges, so adding up the edges connected to each vertex gives \(n(n-1)\) edges. But each edge is connected to two vertices, so adding up the edges this way counts each edge twice. This means that the complete graph on \(n\) vertices has \(\frac{n(n-1)}{2}\) edges.
When you color each edge in a complete graph on \(n\) vertices red or blue, you have two choices of color for each edge. So, the total number of ways that you can color the edges of the graph is:
$$ \underbrace{2 \times 2 \times \cdots \times 2}_{\frac{n(n-1)}{2} \text{ times}} = 2^{\frac{n(n-1)}{2}}. $$
That's two to the power of \(\frac{n(n-1)}{2}\). In the best case — that is, if \(R(5, 5)\) is forty-three — you need to check 2^{903} edge colorings to verify that every single one has at least one red 5-clique or one blue 5-clique.
2^{903} is roughly 10^{271}. For comparison, astrophysicists estimate that the universe contains about 10^{80} atoms!
The number of edge colorings you'd need to check to compute \(R(6, 6)\) is even more staggering. At best, you'd have to check 2^{5151} — roughly 10^{1550} — edge colorings!
For the last half-century, mathematicians have spent significant resources checking edge colorings of the complete graph on forty-three vertices. Each edge coloring checked has contained either a red 5-clique or a blue 5-clique. There is a consensus that \(R(5, 5)\) is likely equal to forty-three.
But why haven't we found the value yet? Erdös posed his hypothetical scenario involving alien invaders in 1990. He thought that \(R(5, 5)\) could be found within a year, assuming the fastest computers and brightest minds were working on the problem.
Since 1990, computer processors have sped up exponentially, thanks in part to Moore's Law. This "law" states that the number of transistors on a microchip doubles roughly every two years.
If we assume that this means that processor speeds also double every two years, then modern processors in 2021 are approximately 2^{10} times faster than processors in 1990. Under these assumptions, a task that took a year to compute in 1990 should take about eight-and-a-half hours to compute today.
Of course, these are just rough estimates. They don't consider other limiting factors, such as interest in the problem and access to sufficient computing resources. And — perhaps most importantly — we haven't had an alien force threatening to obliterate us if we don't compute \(R(5, 5)\) quickly enough.
Given the exponential increase in processing power, though, and the prior work on \(R(5, 5)\), it seems feasible that we could meet the aliens' demands, perhaps in much less time than a year.
What about \(R(6, 6)\), though?
To give you some idea of how large the number of edge colorings that need to be checked is, consider that the universe has been around for about 13.77 billion years. There are 86,400 seconds in a day and, using the Gregorian year of 365.2425 days, a total of 31,556,952 seconds every year.
This means that approximately 4.34 \(\times\) 10^{17} seconds have elapsed since the big bang. That's 434 quadrillion seconds!
Using the best lower bound for \(R(6, 6)\), the best-case scenario means checking approximately 2^{5151} edge colorings. That's nearly 9.3 \(\times\) 10^{1532} times more edge colorings than seconds since the beginning of the universe!
Even if you could check one edge coloring every nanosecond, it would take you more than 2.94 \(\times\) 10^{1516} years to check them all! And, from the looks of it, you'd need a computer with more processors than there are atoms in the universe to leverage parallel computing to knock the time needed down to a year.
Not even quantum computing offers much hope. The best algorithm for the kinds of search needed to sift through edge colorings is called Grover's Algorithm, and it can search through a space of \(N\) items in about \(\sqrt{N}\) steps.
In other words, searching through the 2^{5151} possible edge colorings of the complete graph on six vertices would take about 2^{2576} steps. Assuming each of those steps took about a nanosecond, you still need about 9 \(\times\) 10^{758} years to search through all of the edge colorings.
The numbers are mind-numbingly huge. And while it's true that not all edge colorings need to be checked — after all, mathematicians have found ways to rule out certain configurations — the scale of the numbers is so massive that, even using our best computers, we'd need an earth-shattering breakthrough in mathematics if we are to find \(R(6, 6)\) in less than a year.
Let's hope Erdös's alien invaders never pay us a visit.
Was this article helpful or enjoyable to you? If so, consider supporting me. Every little bit helps motivate me to keep producing more content.
If you'd like to get notified whenever I publish new content, you can subscribe via RSS or join my email newsletter.
Many thanks to Tariq Rashid for reading an early draft of this article and providing valuable feedback.
If you find an error in this article, you can report it here.
Angeltveit, V., & McKay, B. (2018). R(5, 5) ≤ 48. Journal of Graph Theory, 89, 5– 13. https://doi.org/10.1002/jgt.22235
Graham, R., & Spencer, J. (1990). Ramsey Theory. Scientific American, 112–117.
Greenwood, R., & Gleason, A. (1955). Combinatorial Relations and Chromatic Graphs. Canadian Journal of Mathematics,7, 1–7. doi:10.4153/CJM-1955-001-4
McKay, B., & Radziszowski, S. (1997). Subgraph Counting Identities and Ramsey Numbers, Journal of Combinatorial Theory, Series B, 69, Issue 2, 193–209. https://doi.org/10.1006/jctb.1996.1741
Radziszowski, S. (2011). Small Ramsey Numbers. Dynamic Surveys. Electronic Journal of Combinatorics.
The diagrams and figures in this article were created with Canva and Excalidraw.
]]>Imagine meeting an extraterrestrial civilization and, for the sake of argument, let’s assume that you can somehow communicate with these beings. This civilization is a peculiar one in that it has no concept of what we call mathematics.
During some friendly conversation, you mention something about math.
Your curious alien companion stops you and asks, “What is this ‘mathematics’ that you speak of?”
“Well,” you begin, “mathematics is the study of…”
You pause.
What exactly is mathematics? How do you describe it?
This question has recently plagued me, and I fear I have no good answer. Fortunately, I’m not alone. Even Wikipedia recognizes that mathematics “has no generally accepted definition.”
As any good digital citizen would, I turned to Twitter to crowdsource an answer:
I didn’t put too much thought into my poll choices.
I chose Numbers because whenever I told people that I was studying mathematics, one of the most common responses was, “Oh? You must be really good with numbers!” (I’m not.)
Logic was meant to be a bit of a red herring. Mathematicians certainly use logic as a tool. But so do theoretical physicists and philosophers. Logicians study logic, and while there have been many logicians that study mathematics, the two pursuits have always been separate in my mind.
Finally, Quantitative ideas was a choice I threw in because it sounded reasonable. It’s general enough to capture a wide range of topics but still keeps the focus on numbers. This isn’t entirely accurate, though. Ask a topologist if their work is quantitative, and they’ll probably say, “Not really.”
To my surprise, Logic and Quantitative ideas received the same portion of votes. Those who answered Other commented with notions such as space, structure, and patterns.
What I hope that poll participants realized is that, to some degree, mathematics studies all of these things. Yet every answer to the question “What is mathematics the study of?” feels inadequate.
While pondering the definition of mathematics, one thing that struck me is that it seems possible to complete this exercise for other scientific disciplines. For example, you could say that physics is the study of how the universe works. Chemistry is the study of what the universe is made of. Biology is the study of living organisms.
I’d be a fool not to point out that these “definitions” are simplistic. Every human endeavor is fraught with nuance. But, to a certain extent, you can somehow distill these disciplines into a single sentence. This doesn’t appear to be the case with mathematics.
Throughout history, mathematics has been placed on a pedestal and given an almost god-like significance. One example of this is the oft-quoted article titled The unreasonable effectiveness of mathematics in the natural sciences penned in 1960 by physicist Eugene Wigner.
In his article, Wigner expresses his awe of mathematics’ applicability to physics:
“The miracle of the appropriateness of the language of mathematics for the formulation of the laws of physics is a wonderful gift which we neither understand nor deserve.”
Despite my love for mathematics, I find Wigner’s attitude a bit strange and borderline fanatical. In my mind, it’s not surprising that one can describe the observable universe in the language of mathematics. After all, mathematics was born in human minds attempting to understand the things they observed.
I’m no historian, but from what I’ve gathered, the advent of mathematics stems primarily from the need to measure quantity. As civilization formed and trade and economies developed, we humans needed to keep track of everything from land area to amounts of goods to currency.
Humans are an inquisitive bunch, though, and our capacity for abstraction is not a modern development. You can find examples in Indian texts from as early as 800 BCE of what we now call the Pythagorean Theorem.
The concept of number itself is a powerful abstraction. We don’t know who the first human was that recognized that three grains of sand and a group of three friends gathered together to have a shared three-ness. But we know that at some point, it did occur — likely a very long time ago and, I would guess, to multiple people independently.
What is clear to me, though, is that mathematics began with observations of our world. Necessity drove the invention of numerical systems and measurement. Curiosity carried the invention to new heights.
Greek mathematics is widely considered the birthplace of mathematical proof. Most of the accounts of the history of mathematics I've read put the Greeks at the center of modern mathematical thought. Whether or not this narrative is accurate is up for debate.
The story goes that Thales of Miletus learned mathematics from the Babylonians and Egyptians. Later, Thales brilliantly applied logic and reasoning to explain mathematics earning him the title of the first person to apply deductive reasoning to geometry.
Some modern scholars are skeptical of the Babylonian influence on Thales’ mathematics. But the Babylonian and Egyptian influence on Greek society is hardly controversial. And, as pointed out by Karine Chemla in her prologue to the book The History of Mathematical Proof in Ancient Traditions, “several reasons suggest that we should be wary regarding the standard narrative.”
For one, it appears that sources have gone missing from the historical record. Chemla offers as one piece of evidence how remarks made by Henry Thomas Colebrooke, the first person to translate ancient Sanskrit texts to European languages, seem to have been ignored by later historians:
“Colebrooke read in the Sanskrit texts a rather elaborate system of proof in which the algebraic rules used in the application of algebra were themselves proved. Moreover, he pointed resolutely to the use in these writings of ‘algebraic proofs’. It is striking that these remarks were not taken up in later historiography. Why did this evidence disappear from subsequent accounts?”
So, Indian mathematicians — and quite possibly Babylonian and Egyptian mathematicians — were doing mathematical proofs before the Greeks. They just didn’t look like Greek proofs, and historical bias has all but erased their contribution to this part of the development of mathematics.
What we can gather, though, is that at some time, mathematics moved from accounting and engineering to something more like philosophy. The statement of a mathematical principle must be accompanied by a proof explaining why that principle is true.
So what, then, is mathematics? Mathematics has exploded into something far beyond the geometry and algebra handed down to us from the ancients. Some of the objects studied by mathematicians are so abstract that they have no apparent example in nature.
This is where I have to come clean. I really don’t know what mathematics is. I know a few things that it is not, and a few things that it is in part.
The most significant difference I see between mathematics and sciences like physics and chemistry is that mathematics, although rooted in observations of our universe, is not concerned with material objects.
So perhaps mathematics is best described as the science of the immaterial universe — the universe of human thought. Just as physical science is divided into various disciplines — like astronomy or chemistry — so too can mathematics be broken down into things like geometry, algebra, and analysis.
In other words, I wonder if the word mathematics is more akin to the word science than it is physics. It is not a discipline in itself but encompasses many disciplines that share a common toolkit. Science has the scientific method, and mathematics has deductive logic.
Doesn’t this place mathematics into the realm of philosophy? Perhaps. But even philosophers struggle to explain mathematics. Stewart Shapiro, in the Oxford Online Handbook’s introduction to the philosophy of mathematics, has this to say about the trouble mathematics presents to philosophers:
“The conflict between rationalism and empiricism reflects some tension in the traditional views concerning mathematics, if not logic. Mathematics seems necessary and a priori, and yet it has something to do with the physical world. How is this possible? How can we learn something important about the physical world by a priori reflection in our comfortable armchairs? As noted above, mathematics is essential to any scientific understanding of the world, and science is empirical if anything is—rationalism notwithstanding.”
Is it this duality of rationalism and empiricism that plagues anyone who dares try to define mathematics? Maybe. I don’t know. I’m not sure I’ll ever know.
There is one thing I can say now that I’ve spent a couple of weeks thinking about the definition of mathematics. The ineffectiveness of any definition now seems to me to be perfectly reasonable.