Introduction
Every now and then I get asked by students in the computer engineering department, which I head in my institution, why a particular course is important for our major. I typically get this comment about courses from outside the department that serve to provide supporting knowledge for computer engineers. I have tried to explain the importance of these courses so many times, that I think it would be best to just create a series of blog posts about why they are important and direct students to those links when I get asked these questions.
The most common courses I get asked this question for are
- Maths courses — including calculus, probability and statistics, special functions, fourrier series and similar, numerical methods and so on.
- Physics courses — including the usual Newtonian physics up to relativity and quantum mechanics.
- Automatic control courses — typically taught to us by the Electrical Power department.
- Electronics — typically taught to us by the Electronics and Communications department.
- Communications theory — typically taught to us by the Electronics and Communications department.
In this, the first post in this series, I will try to explain why fundamental knowledge is important — why it is important to know how something works and to have sufficient skills to be able to go into things and change them if required. In subsequent posts, I will take each of these courses individually and try to explain why they are important for computer engineers. Hopefully, I will be able to have a set of posts that I can share with students whenever I get asked about any particular course. I also hope it will be useful for the general reader.
Isaac Asimov — The Foundation Series
One of my favorite authors is Isaac Asimov. While his novels may not be masterpieces of character development like the classics of literature, they do provide a very strong critique of the way societies tend to decay and decline. In particular, his Foundation series of novels revolve around the idea of how an advanced society can fall into decline.
I believe he mentioned somewhere that he was inspired by The Rise and Fall of the Roman Empire, itself a masterpiece in this area, but I was more inspired by his books, perhaps because they are an easier read?
Anyway, regardless of the reason that my imagination got inspired by these books, I remember reading them as a young adult and being impressed about how he describes a society that falls from the height of technological advance into disarray. Of course, I was also excited by the way the protagonists overcame difficulties and created the germ of a new civilization that grew out of the ashes of the old, but my mind was captivated by how he described the decline of the old Empire.
One of the reasons he mentions is the decline of learning. People in the old Empire become so dependent on their machines, that they forget to do original research. The machines became black boxes that they could barely maintain, let alone innovate new things. The series of books uses nuclear technology, called atomics in the books, to represent the latest in technology — that was probably true at the time the books were written. But the core idea remains the same for any technology.
He even mentions the same concept when discussing archeology. I remember a scene in one of the books when a visiting dignitary expresses his interest in archeology, and tells the protagonist that it is useless to do any field work, rather it is sufficient to study what previous researchers thought, and to come to a conclusion based on that — the concept of a survey paper being elevated to the height of research to the exclusion of everything.
All this amounts to one thing, if a society treats tools as black boxes, and forgoes original research, it will surely fall into decline. As the generations with knowledge about how to build things and to innovate dies, the new generation will be helpless as their old machine eventually fail and they have no knowledge to maintain or build new ones.
What, you might ask yourself at this point in time, has this got to do with the main topic of this post? What has this got to do with students asking about the importance of certain courses — always with the undertone of “these courses are not important, and we don’t get good grades in them, so lets skip them or remove them from the curriculum”. Let us now try to answer this question.
So what has this got to do with anything?
The courses that the students dislike, and that some parents actively try to convince the department to remove, provide foundational knowledge in engineering. They provide core concepts that allow us to build computer systems and programs that are functional and meet requirements. If we do not learn them, we will be like the citizens of Asimov’s old empire, using our machines without really knowing how they work — worse, without knowing how to maintain or build new ones. We will be taking the first steps towards civilizational decline.
The maths courses provide the basics of the mathematical foundation that underpins all computer knowledge. From linear algebra to differential equations to probability and statistics, these are the core tools we use to build meaningful programs.
I will dedicate separate posts for each course, but think how impossible it would be to build machine learning tools without knowledge of probability and statistics, or how it would be impossible to build a physics engine for game development or computer generated graphics for movies without the knowledge of Newtonian physics to specify the laws of physics, the knowledge of calculus to mathematically represent the laws, and numerical analysis and linear algebra to numerically represent these laws inside the computer.
How impossible it would be to design cryptography algorithms without the knowledge of number theory, how impossible it would be to build and program quantum computers without knowledge of quantum mechanics, how it would be impossible to design accurate GPS systems without at least a basic knowledge of general and special relatively — the list goes on.
All these subjects provide core knowledge that allow us to build software and hardware that work. At this point, some students point out that they can just use existing libraries to get the functionality they want, without knowing the details. But that is just what the old empire did in Asimov’s books — they used things without knowing how they worked. What happens when the library doesn’t do exactly what you want and you need to get into the code to add support for it?
What happens when the library becomes outdated, as all software eventually does, and a new version needs to be written? Who is going to do that? If we rely on things that have already been designed and use them as black boxes, our intellectual capabilities will atrophy and we will be destined to the fate of Asimov’s old empire, or to the fate of the Roman Empire, or any other empire in history which fell into decay by becoming decadent and forgetting the importance of core knowledge.
By the way, this goes beyond just reluctance to study courses, it also covers the wrong use of modern tools. If we rely on these tools completely, we will also go down the path outlined above.
How modern tools can have the same effect
First, let me clearly state that I am not a Luddite — far from it, I am one of the strongest supporters of technology. I’ve been using programmable calculators and home computers ever since they came out — and when the latest generative AI technology came out, I jumped on the bandwagon and I am even planning to incorporate them into my courses — after a brief period of being wary of this.
But it is important to note that these are tools, they are not replacements for creative and innovative thinking. For example, if you use chatGPT to replace you as a writer, or do your homework, or if you expect Github Copilot to write all your code for you, you are not really using these tools correctly.
Sure they can produce amazing results, but you need to drive the creative process. These tools are not capable of coming up with, for example, novel programming solutions. They are, when you get right down to it, stochastic models that can predict the next word or code element based on their training sets. So they are very good at restating what has happened before, in a way that has, probably, never been said before.
But they cannot truly innovate. Even the AI art generators, like Midjourney or Dall-e 2, are merely producing output that matches the probability distributions of their training sets. So they can come up with new art in the style of existing artists, or even combine styles, but they cannot come up with something truly new. They cannot produce new art forms, like the Surrealists did or the Impressionists. They produce new material that matches the style of their training dataset.
What does this mean? It means that we should embrace the use of these tools to automate the routine parts of knowledge work, like producing boilerplate code to perform a certain function given a particular set of data declarations, or to outline a report on a topic that has already been addressed before. But it would be impossible to expect such models, for example, to write articles about a technology that has just appeared or a world event that has just happened — these are simply not in their training sets.
When these events or technologies get written about sufficiently on the internet, they can be incorporated into the training set of the generative models. When this happens, they can be used to summarize and explain these concepts or events concisely, but they cannot handle new material or come up with new ideas. They may hallucinate false information when faced with something they don’t know, but they cannot come up with new concepts — at least not yet. They are good at what computers have always been good at, processing large amounts of existing information and finding patterns in it that they can exploit.
So what does this mean? It means if we rely on these tools blindly, without verifying facts and without retaining our abilities to think critically or to solve problems from fundamental principles, we will also be going down the road of Asimov’s imagined civilizational decline.
Conclusion
It is very important to have core knowledge of the foundation of whatever field you are working in. Anything that disrupts this can cause a decline in a society. Whether this disruption comes from ignoring fundamental knowledge in the belief that it is irrelevant or from relying on automated tools that do stuff for us, if we let our core knowledge die and our ability to think critically atrophy, then we are on a slippery slope to a state where we are incapable of maintaining society at its current level, let alone advancing it forward.
