Seven Fields of Computer Programming: A Brief Survey

Written July 7th, 2012, Revised Jan 12th, 2013, Updated & extended Jan 25th, 2020

There are at least seven distinct fields of computer programming that have less to do with languages and more to do with the target platform, desired functionality, and intended user. This article provides a short introduction to each, intended as a brief orienting survey. These are:
(1) Bare metal programming, not requiring an operating system,
(2) Application programming, in which an operating system is presumed,
(3) Mathematical computing and algorithms, from matrix computations and statistical learning to wavelet compression and cryptography,
(4) Web or Client-Server programming, in which the application lives in a client browser in communication with content generated on-the-fly from programs running on central servers,
(5) Mobile or App programming,
(6) Cloud programming, and
(7) Exotic programming (traditional super-computing, quantum computing, biological computing/soft robotics, deep machine learning).

1. Bare Metal Programming

Bare Metal programming works at the level of the machine, without the assistance of an operating system. This is often called bare metal programming. It is most typically needed for low level embedded controller programming, hobbyist applications, perhaps robotics, or tiny devices. The languages could be anything from machine language, assembly language, C compiled down to an embedded target, or even a higher level language interpreted language such as Forth, Basic, or Python. The key is that the generated code would make no assumptions about an operating system, and would be designed to work on the machine directly.

Programs in this space typically don’t work heavily with files. They might be device drivers, tight embedded systems, they may have I/O typically I/O lines or A/D streams from sensors. Or they may work with packets over a network. Constraints might be the requirement to keep up, also called a real-time requirement, i.e. to do all of the processing within the cycle time to the next event, which could be minutes away or milliseconds away (the latter is more a hard real-time requirement, where the former is a softer real-time requirement). There may be memory, I/O, networking, and display constraints. These types of programs are rarely applications — they are usually kernels, operating systems, device drivers, embedded control logic, or other low level components that will be used to support applications or that deliver functionality that typically doesn’t require complex user interaction. This type of programming has been around since the very beginning, but saw its hey-dey in the 1970s and 1980s, before the dominance of OSes, at which point it became a specialization.

You can play around with Bare Metal Programming using any Arduino that uses the Atmega328P chip (Uno or Nano will work), and 328eForth by C.H.Ting, which gives you a Forth interactive environment from which to explore the chip — no operating system in between, just the tiny 5k Forth virtual machine sitting in the Atmega328P’s bootloader. See the resources in Forth section of Additional Reading, below. Articles on this site are “Hurling Boulders (Assembly 1)“, Assembly for Embedded Systems (Assembly 2), “Assembly Toolkit (Assembly 3)“, “Bare Bones (C)“, and Electronics Gateway to Technology.

2. Applications Programming for Desktop

Unlike bare metal programming, applications programming typically presumes an operating system such as Windows, Linux, iOS, Android, or VM (Forth, Java, etc) sits between the machine and the user level program. Here, the program typically interacts with the user via the operating system which controls access through keyboard, mouse, or touchpad, via graphical controls, and displays results on a monitor or stored in files. Input is received from the user or read in from files, all mediated by calls to the operating system, which abstracts the peripherals and computer architecture from the programmer through various interfaces (APIs): file interface, memory interface, display interface, keyboard / mouse interface, and graphical control elements. Your job as a programmer of such a system is to understand the capability of the operating system, and weave its elements together into a program that delivers some useful functionality, in a way which is intuitive, easy to understand, and easy to use. There are typically fewer constraints than on the other two kinds: memory is typically plentiful, display area is typically large. The most significant constraint in this kind of programming is usability and robustness: clarity, intuitiveness, and ability to perform in the presence of all manner of possible user errors. These types of programs are typically user applications that often require complex user interaction. This type of programming began in earnest with the development of hegemonic operating systems in the late 1970s and through the 1980s: CPM, OS/2, MS-DOS, Apple II, Multics and Unix. It became mainstream in the 1990s.

There are a number of fundamental changes in programming methodology that emerged with the rise of applications programming, and the search for the ideal machine in silicon.

(a) structured programming (Niklaus Wirth, inventor of the Pascal language,

(b) the mathematics of softare reliability (Edsgar Dijkstra, critic of laissez-faire practices in mainstream software engineering and content in computer science education),

(c) programming a problem-orientated language (Chuck Moore, inventor of the Forth language and proponent of minimalism in code and silicon), (see Article here),

(d) object-oriented framework (Alan Kay, inventor of the Smalltalk language), and

(e) the foundations of computer science upon discrete/concrete mathematics (Don Knuth, the Art of Computer Programming).

See this discussion for influence between Chuck Moore and Don Knuth on algorithms and the search for the ideal machine.

(f) the search for the ideal machine in silicon: CISC (complex instruction set computing) vs. RISC (reduced instruction set computing) vs. MISC (minimal instruction set computing) (Intel & Microsoft, promoter of CISC; David Patterson, Sun/SPARC promoter of RISC; Chuck Moore, promoter of MISC)

You can play around with Applications Programming for Desktop using almost any language, but with a modern OS you have restricted access to the I/O devices of the computer. That changes if you use MS-DOS, which you can run within DosBox VM or vDosPlus. Feel free to explore this fully-fledged Turtle Logo program I wrote for Windows using F-PC Forth (“Coding for Pre-Schoolers”). It is just 900 lines of code (< 1Kloc) and illustrates the expressive power (and succinctness) of Forth for building application programs using the problem-oriented language approach.

3. Mathematical Computing

Mathematical computing was perhaps the original motivation for the development of computing machinery, with large linear systems of economic equations, ballistic trajectory calculations, and operations research problems among the early problems tackled by the first generation mainframe computers (switches, vacuum tubes, punch cards).

Innovation has driven mathematical computing alongside theoretical advancement and numerous groundbreaking applications, much of it in major research institutions or industrial research labs such as AT&T Bell Labs, IBM, and Lucent, much funded by governments (NSA, DoE, DoD, NSF, etc.)

(a) Assemblers (Grace Hopper, inventor of COBOL language),

(b) Theoretical limits of computers and computing design (Alan Turing inventor of the Turing machine, John von Neumann inventor of the shared memory architecture between programs instructions and data, Alonzo Church inventor of the lambda calculus which would lead John McCarthy to invent LISP, much of this a back-and-forth between mathematical logic, philosophy, and computing — see here for the stories behind 30 great names),

(c) Parsers, lexers, and compilers bringing linguistic theory into computer science (John Backus inventor of FORTRAN and the Backus-Naur form for precisely representing languages)

(d) Algorithms and complexity (Don Knuth for Art of Computer Programming multi-volume comprehensive mathematical treatment of algorithms from first principles, Nijenhuis, Herbert Wilf and Doron Zeilberger for combinatorial algorithms, generating functions and hypergeometric series to transform combinatorial identities, Nash, Godel, and von Neumann for P/NP hardness classifications)

(e) Computer Algebraic Systems (CAS) (Stephen Wolfram of Mathematica, Joel Moses and others for Macsyma and William Schelter for Maxima)

(f) Matrix and Numerical Computing (Cleve Moler inventor of Matlab, John Eaton for Octave, and for theoretical advancement Richard Hamming of Bell Labs, Golub, Van Loan, Wilkinson, Demmel, Higham, Trefethon, Greenbaun, Strang, Horn and Johnson, LeVeque)

(g) Optimization packages (Dantzig for combinatorial optimization, Karmarckar for linear programming, Rockafellar for non-linear optimization, Kernighan of Bell Labs for AMPL)

(h) Statistical computing (John Chambers of Bell Labs for S, Gentleman & Ihacka for R)

(i) Expert systems (Ed Feigenbaum of Stanford)

(j) Machine (Statistical) learning, starting with pattern recognition, and then supervised and unsupervised learning, PCA/ICA, amongst other techniqes (Duda and Hart, Karl Pearson of PCA)

(k) Fuzzy learning (Lotfi Zadeh of Stanford)

(l) Parallel computation and parallel solvers for large linear systems (Dongarra for MPI, Ercal and Okunbor for applications)

(m) Information theory and mathematical compression (Haar, Gabor, Meyer, Mallat, Daubechies for wavelets, Claude Shannon of Bell Labs for information theory and entropy)

(n) Cryptography and encryptian (Rivest, Shamir and Adleson for RSA public-key encryption system, Nakamoto for bitcoin)

All of these areas continue to have active research communities, are highly mathematical, and their break-throughs and applications continue to shape our technological society.

Articles from this site are: Finite Summations of Integers (1-Mathematical, 2-Maxima, 3-Octave), Maxima for Symbolic Computation, Mathematics of Duelling (R simulation).

4. Web Programming

Web programming took off in the 2000’s, with focus on programming for a container such as a web browser, with a client and a server, perhaps Mozilla, Chrome, or Internet Explorer. The complexity here is trips between server and client. Ajax, introduced by Google, transformed what is possible on the client side.

You can play around with CGI, running a virtual server on your desktop and writing client web software against it. For more advanced exposure, you will need to look at JavaScript, Ajax, Ruby on Rails, or Python’s Django kit.

5. Mobile/App Programming

Mobile/app programming took off in the 2010’s, driven by the rise of iPhone and Android smartphones, and focused on programming for a mobile environment with limited peripherals, small but high res displays, and touch interfaces. Here the target is a mobile phone, a tablet, or a wearable computer of some kind. There is typically a lightweight operating system managing the interfaces to the device. But there may be constraints that are not found on larger computers. Memory might be tight, the computer might be small and slow, there may be a constraint on network bandwidth, the display area might be small, and input might come through gestures or through voice commands. These types of programs can be of both types, and may use networked resources (cloud storage, location awareness, cloud processing) and built-in sensors (GPS, gyroscope, accelerometers, blue-tooth, wi-fi) to deliver unique content through clever integration.

You can play around with Bernd Paysan’s GForth for Arduino.

6. Cloud Programming

Cloud programming took off in the mid-2010’s, as a vast server infrastructure began to be build by Amazon, Microsoft, Google, and Facebook to drive their web and mobile platforms. The focus is on programming for/within the so-called “cloud”, with toolkits and devices from AI stacks, and complete virtualized environments with Amazon’s AWS and Microsoft’s Azure as the front runners for individuals and businesses.

You can play around with AWS’s stack, paying for computing resources as you go.

7. Extreme Programming

Every decade since the invention of the programmable computer in the 1940s has seen its extreme programming environment. Currently (in the 2020’s), these are:

(a) traditional super-computing,

(b) quantum computing,

(c) biological computing/soft robotics, and

(d) deep machine learning (Dennis Hassabis of Deep Mind, and 23 current groundbreaking researchers).

Hard to play around, but some fun reading on what’s happening in these spaces. “Knowledge Engineering” is a concept map from the 2000s.

Additional Reading

The following provide a guided journey through key topics above — work in progress.

A. Systems & Platforms
1. CISC vs. RISC vs. MISC, MuP21: A High Performance MISC Processor (MuP21 = MISC Microprocessor 21) C.H. Ting & Chuck Moore, 1995
2. Don Knuth’s MIX and MMIX, a hypothetical RISC machine
3. NOVIX, Chuck Moore’s stack-based compupter, a MISC machine realized in silicon
4. FORTH: A Language for Interactive Computing, Chuck Moore’s abstraction of any computer into a stack machine
5. Y86, a hypothetical CISC machine (stripped down Intel 80×86). Specification, Bryant & O’Halloran
6. Verilog & HLC implementations of Y86 CPU
7. Why teach x86 (CISC) rather than RISC? (Bryant & O’Halloran). Introduction to CISC CPUs, Learning Intel 80×86 assembly language programming
8. Recommended Textbook: Computer Systems: A Programmer’s Perspective (3rd edition), 2015, Randal Bryant and David O’Halloran, Carnegie Mellon University. 3rd ed. Textbook page, Careful to get the US edition, not the global edition which has many mistakes 1st ed. (2002, US), £6.80, Amazon.co.uk

“The idea with [this course approach] was to introduce students to computers in a different way. Few of our students would have the opportunity to build a computer system. On the other hand, most students, including all computer scientists and computer engineers, would be required to use and program computers on a daily basis. So we decided to teach about systems from the point of view of the programmer, using the following filter: we would cover a topic only if it affected the performance, correctness, or utility of user-level C programs.




Topics in 3rd ed. of Computer Systems: A Programmer’s Perspective, by Bryant and O’Halloran (2015)

B. Forth

9. Scot Stevenson, How I came to join the underground computer cult of Forth, (Feb 20, 2014)
10. Why Forth? By Bernd Paysan, creator of GForth and GForth for Android
11. The Dragon-free Internals of Forth, Scot Stevenson, (May 17, 2014)
12. The Joy of Forth: Rediscovering our past, Jeff Schilling, 2007
13. Forth postings, Scot Stevenson, 2014/2015 posts
14. Programming in Forth – Books from Forth Inc. (many free)
15. Forth Library on Amazon
16. Arduino Controlled by eForth, C.H.Ting, 2018, PDF available for free. Explains how Forth lets you learn a chip interactively – bare metal programming!
17. Forth with Life (integrated Mobile/Cloud medical application).
18. [Moore/1970a] Forth: A Language for Interactive Computing, Chuck Moore, 1970, PDF download
19. [Moore/1970b] Programming a Problem Oriented Language (POL), by Chuck Moore, 1970, (PDF download)
20. [Moore/1991] Forth: The Early Years (History of Programming Languages), by Chuck Moore, 1991. In it, he describes the 10 years it took for him to invent Forth (1958-1968), between MIT’s Smithsonian Astronomical Observatory (he knew John Gaustad, took a course from John McCarthy on LISP, and had looked at APL), Stanford, and Mohasco.
21. The Evolution of Forth, Rather, Colburn, Moore, 1993
22. CISC vs. RISC vs. MISC, MuP21: A High Performance MISC Processor (MuP21 = MISC Microprocessor 21) C.H. Ting & Chuck Moore, 1995
23. The Continuation of Chuck Moore’s Journey in search of Computing Nirvana (my title, Jeff Fox’s main page of Ultra Technology)
24. Forth-79 standard
25. Forth-83 standard Word List Summary
26. ANS-Forth 94 standard
27. Threads of a FORTH Tapestry, Byte, August 1980, Editorial
28. Go FORTH and multiply!, 1981 article
29. The Many Forths, Survey by Jeff Fox. TinyMicros Forths
30. Forth Chips, Survey by Jeff Fox. FIGnition DIY computer for £20 on Ebay
31. A 3-instruction Forth
32. GreenArrays Technology, GA144 – 144 full-fledged processors on a single chip. People
    

    ---

    Z. Giants of Computing

33. Dennis Ritchie (d. Oct 12, 2011, aged 70), Father of the C Language. In Memoriam by Bryant & O’Halloran
34. Charles (Chuck) Moore (b.1938), Father of Forth, and Stack-based Computing. Wikipedia. 22 patents (Moore Microprocessor Patent library)