DHQ: Digital Humanities Quarterly
Editorial
The Epistemology of Code in the Age of Machine
Learning
Code as an Epistemic Object
Code primarily serves an epistemic function. The specific form of code, then, follows
our changing episteme, our idea of what knowledge looks like and how we can (or
cannot) verify its claims. The code concept comes from an episteme that emerged
around 1930, in which we conceptualize knowledge as primarily a process for the transformation of one representation into another.[1] In this episteme, meaning appears
as either purely definitional — a rule that substitutes one piece of text for another
piece of text — or as the sufficiency of the symbolic representation itself — the
sense that the meaning of textual atoms “properly” derives from their participation
in this system of representations and definitions, and not from anything outside that
system.
Accordingly, code takes form as the posited sufficiency of its representation, the
identity of its representation with the calculative process it represents. That is,
code must define the process it represents, rather than depicting only some aspect
of
that process. Code must both represent and constitute algorithms. But there is nevertheless always a remainder, a part
of every algorithm which is not explicitly represented in the code that is supposed
to constitute it. The code concept is completed through externalizing this remainder
as state. Code then takes on the character of the essential
and fundamental form of the algorithm, whereas state is relegated to the accidental
set of objects with which the algorithm works. Nevertheless, code and state are
consistently entangled, such that state continually reappears within code, and
consequently code appears to contain what is accidental and to lack a complete
representation of what is essential. Code is then reformulated such that this
intrusion of state is more rigorously excluded from the epistemic space of code. The
history of code follows this general structure of externalization and repression,
the
return of the repressed, and the incorporation of this return in a new form of
externalization and repression.
In 2022, with the increasing centrality of global optimization algorithms such as
the
“machine learning” type of artificial intelligence, we stand at the other end of this
history. The valence of code and state
have begun to change places, without as yet a fundamental change to the form of code.
This results in an increasing sense that something important, perhaps the most
important thing, is happening within state rather than code. And yet state is still ideologically unavailable to
knowledge, a nonepistemic object, excluded from code and repressed in its
representations. The nonepistemic nature of global optimization algorithms is now
so
obvious that it has become ubiquitous even in popular culture, where artificial
intelligence is described, for example, as a “black box” that we can “never know”
(kemper, 2019). We are in an epistemic crisis which is a direct result of the
epistemic form of code itself. To understand this crisis, we must examine the history
of this form.
This paper will examine the early history of code as the history of the repression
of
state, using methods of comparative close reading developed in the discipline of
critical code studies (marino, 2020) (marino, 2014). It will then conclude by looking
at the specific form of global optimization algorithms and how these algorithms both
relate to and problematize the ideology of code.
Critical code studies is particularly well suited to the analysis of ideology, the
often unconscious structure of our ideas. The complementary disciplines of platform
studies (montfort, 2009) and software studies (fuller, 2008) generally examine code
and computers as a form of media or as a pervasive technological object. With those
methods we can understand the ways software and computers enable or restrict our
actions, lives, and communications. But critical code studies focuses on another
dimension: meaning. The analysis of language brings with it the concept of meaning
and its conventional connection — we might say ideological
connection — with representation. While signs dwell in language, the ideological
force that composes them is not contained in language. Critical code studies asserts
that there is meaning in code, but just as much that there is
meaning around code. While the concept of code is not defined
through particular pieces of code, nevertheless, through examining the distance
between the idea of code and its actual, representational existence in these
particular instances we can begin to see the outlines of the ideological structures
that compose the code concept. Further, in the specific case of code, because the
intelligible function of digital information is generally subordinated to its
socioeconomic function, the textual materiality of informational structures becomes
a
central focus for economic efficiency. Reading code therefore means seeing code both
in its determination and distance from this textual materiality.
The Formation of the Code Concept
The code concept came into being through the rise of an episteme focused on process. As the pace of capitalism quickened in the 1920s and
30s, the grandiose systems of the 19th century and the mathematical axiomatizations
of the early 20th century no longer resembled our lived relationship with knowledge
and information. In place of the ideology of system, we
developed the ideology of process, the relation of different
pieces of information through their transformation in form and locus. In this
ideology, process is wholly symbolic in both form and content. That is, both the
process itself and the material with which it works are defined symbolically and are
not supposed to have a remainder outside of a symbolic system.
Process and its material are not supposed to have any meaning which cannot be
elaborated through a textual transformation.
Process can become abstract only in so far as it is distinguished from the material
with which it works. To put it another way, the idea of an abstract symbolic process
is the idea that some element of process is somehow the same across multiple actual processes which are
differentiated only by their material, their “data,” only by the symbols on which
they perform their operations. However, because both the process and its material
are
symbolic in the same way, the differentiation between abstract process and the
concrete material on which it works becomes an unstable area of epistemic contention.
Handling this instability becomes one of the central tasks of the broadly capitalist
episteme in its phase from circa 1940 to 2000.
This procedural episteme provided the basis for the formation of code, but
information processing was first mechanized before this in a systemic episteme where knowledge was conceptualized as a static, structured
relationship of the parts within a total system. While procedural thinking casts relationship as procedure, systemic
thinking casts procedure as relationship. Accordingly, from the 1890s well through the beginning of the
computer era, punched card processing provided an ecosystem for the processing of
information based around the relationships between individual
data points or unit records, the punched cards on which data
was stored (heide 2009) (campbell-kelly, 2014, 13–40). These relationships were
created and reconfigured through plugging wires on a plugboard, creating a physical
route for data in its progress through the machine. Punched card machines would cycle
as a whole from 9 through 0, and as a number in the cycle occurred, if that number
was present on the punched card an electrical impulse would be delivered to a plug
representing that column, which would then travel through the plugged wire to another
part of the machine that would run calculations or output data on newly punched cards
(ibm, 1956).
When we first see the textual representation of information processing control in
the
early 1940s, it follows this systemic paradigm, and represents the process as the
relationship, through routing data, of different elements of information. This is
true for the code of the 1944 Harvard/IBM ASCC, better known as the Mark I; the 1948
IBM SSEC, its successor; and the 1946 Bell Labs Model V relay calculator. After that,
the procedural episteme takes over, and we see the development of the modern code
paradigm (haigh, 2014). Of these three, the Mark I is the best known. Examining its
programming system, it becomes clear that it is a continuation of rather than a break
from the paradigm of control used in unit record machines.
The following is a multiplication sequence for the Mark I, as it appears in the
manual:
| OUT | IN | MISC. |
| A | 761 | |
| B | ||
| C | 7 |
Table 1.
(Harvard Computation Laboratory 1946, 111)
In the code for the Mark I, instead of an opcode indicating the operations of which
the device is capable, different portions of the device are given addresses, and
routing data between those addresses performs the operations. Code for the Mark I
contains three columns: “out,” “in,” and “miscellaneous.” Addresses placed in the
columns marked [A] and [B] indicate the multiplicands, which can come from any part
of the machine. The number [761] is the address of the Mark I’s multiplier. Lastly,
the result can be placed anywhere else in the machine, indicated by an address placed
in the column marked [C]. The [7] in the miscellaneous column is an instruction to
the machine as a whole, in this case indicating that it should advance to the next
instruction, which perhaps surprisingly was not automatic.
In this code, the multiplier multiplies because that is what it does, not because
it
has been ordered to do so, as would be the case with later code. The only thing the
code needs to do is to connect to it, to “plug” it into whatever devices are holding
the numbers. The only place here where the code begins to be represented as a process is in the miscellaneous [7], indicating that we should
proceed. Because it is addressed to the machine as a whole, this instruction is
seemingly conscious of the general flow of instructions and their relationship as
text with the total calculative procedure. But everything about this [7] tells us
to
ignore it. It is placed last in the list of columns. It never stands alone but is
always an adjunct to a routing operation. And lastly, the only real effect it can
have on the flow of instructions is to continue or, in its absence, halt. The Mark
I
lacked the ability to represent anything but a purely sequential set of instructions.
As such, the Mark I’s code represents a merely incidentally
temporal sequence of connections between a system of
registers and other devices, not yet a process which has taken
over the epistemic function of this system. And yet, with the miscellaneous [7] as
well as a few other minor features of the language, we can see the beginnings of the
representation of process that would emerge just a few years after this machine.
Modern code was formed through two interlocking developments. On
the one hand, the rising episteme, centered on process, demanded the
epistemic self-sufficiency of the representation of process, that is, of code. As
we
will see, this first occurred through the conditional branch statement, a moment
where code reflexively represents its own procedural nature. But although our
epistemic commitments provided the basis for the procedural and symbolic nature of
code, it was further specified by the physical and economic characteristics of
information processing. Code must work both socially and mechanically. Thus, on the other hand, the economic exigencies of production pushed
the maximum possible amount of computational complexity away from the spatial and
into the temporal, giving rise to a purely sequential representation of process.
The first fully general and electronic calculating device, the ENIAC, was developed
in the 1940s in imitation of the calculative activities of groups of human computers
hired to produce tabulated mathematical data as well as the plugboard routing systems
of punched card information processing. While the ENIAC succeeded in wholly
mechanizing the process of calculation, as well as optimizing its speed through the
use of fully electronic circuits (punched card machines were partially mechanical),
it did not do so in an economically viable way. Through analyzing the problems of
the
ENIAC and the economics of computing machines in general, the sequential aspect of
code was created.
In analogy to the parallel divide and conquer approach of a group of human computers,
the ENIAC consisted not of a single monolithic device, but of a large group of
devices, each one specialized to a certain task: 20 accumulators (performing
addition, subtraction, and storage), 3 multipliers, 1 divider/square rooter, 3
function tables (lookup tables for arbitrarily mapping inputs and outputs), and 3
devices transmitting numerical constants. These were “programmed” by plugging them
together, each one emitting a program pulse to the following device, with the whole
ensemble coordinated by a “master programmer” that could implement very simple
looping, such as might be used for successive approximation algorithms. In this way,
the ENIAC spread computational complexity out in space, and this required a massive
amount of hardware. It used around 18,000 tubes (as opposed to 700 to 3,000 for its
immediate successors) (haigh, 2016, 314 n. 68), each tube adding to the cost and
failure rate of the machine. While the ENIAC was blazingly fast when it worked, far
outpacing any other machine of its era, in its first years, it spent the majority
of
its time being repaired or being set up, and only a small portion of that time
running actual calculations (haigh, 2016, 121).
Because at the time an electronic computer could operate at a speed beyond the
information processing needs of nearly any institution (with a few notable
exceptions), actual computing time was relatively cheap, while the spatial
organization of calculative elements — the computer itself — was extremely expensive.
The solution was therefore to reduce all physical elements to the bare minimum, with
only one instance of each element present, and to reuse these elements as much as
possible during different parts of the calculation. First, reusing the same hardware
in different ways at different times would mean that complexity would take form in
temporal rather than spatial organization. The events which took place on these
calculative elements were organized sequentially rather than
relationally. Second, the nonduplication of data storage
devices meant that code and data would have to share a memory space (although
ideological concerns also led to this requirement). In the design documents for the
successor of the ENIAC, J. Presper Eckert is explicit about all of these developments
and continually emphasizes the specifically economic reasons for the resulting
choices (eckert, 1945). It is important to note, then, that the “objectivity” of
these improvements is social. According to arbitrary measures abstracted from their
social context, such as baseline speed of calculation and perhaps also the intuitive
systematicity of code, modern computer architecture is a regression from the earlier
design of the ENIAC.
From these technoeconomic concerns, we get the purely temporal and sequential nature
of code. But code is further specified through an epistemological commitment to the
purely symbolic representation of process. The sequential form of control, then, must
be represented symbolically without remainder. When code came on the scene, data
already had a representational form (the digital), and the specific mathematical
operations which code performed were already well defined. What remained to be
defined was the reflexive representation within the process of the sequence of the
process itself. As we will see, however, at first this reflexivity was only achieved
through an explicit reflexivity: the process calculated changes to the text of the
program. That is, code was first conceptualized as inherently self-modifying. But
this leaves a remainder, since with self-modifying code the developing process can
only be comprehended through the context of the actual progress of the calculation
over time and as such is not fully contained in the representation of its code.
This self-modifying coding system was never implemented but appears in John von
Neumann’s “First Draft of a Report on the EDVAC” (von-neumann, 1993), a widely
influential document that formed the basis for several early computers. In the “First
Draft,” von Neumann sets out an essentially modern coding scheme, with one important
exception: In the place of a conditional branch — an instruction which transfers
control to some other part of the program based on the result of some operation —
von
Neumann has a conditional substitution. This is despite the
fact that the use for this operation which von Neumann has in mind is not
substitution itself but alteration of the flow of control in the progress of the
program. Von Neumann writes:
“A further necessary operation is connected with the need to be able to sense the
sign of a number, or the order relation between two numbers, and to choose
accordingly between two (suitably given) alternative courses of
action. It will appear later, that the ability to choose the first or the
second one of two given numbers,
u
,
v
depending upon such a relation, is quite adequate to mediate the choice
between any two given alternative courses of action.…Accordingly, we need an
operation that can do this: Given four numbers,
x
,
y
,
u
,
v
, it “forms”
u
if
x
≥
y
and “forms”
v
otherwise.” (von-neumann, 1993, 24)
This would then appear in the notation of the “First Draft” as:
[A ← yA ← x-hA ← uA ← vs → a]
Where in an actual program the italicized text would be replaced by numerical memory
addresses. In the first three lines, [x] and [y] are loaded (the [A] ← instruction)
and subtracted (“-h”), yielding a positive or negative number. Following this, [u]
and [v] are loaded, and the [s] → command chooses between them based on whether the
previous result was less than or greater than [0], which is equivalent to testing
if
[x ≥ y]. Location a is the destination into which the substitution would be made.
Generally, the destination would be the address of an unconditional transfer
instruction, thereby achieving control over the textual flow of the program by
modifying its code.
In the first actual implementation of this coding scheme, a conditional branch
instruction was used instead of a conditional substitution (clippinger, 1948, 21).
Rather than making a comparison and substituting one or another number into the code
based on its results, the conditional branch mada comparison and, depending on the
results, the computer used a different address for the location of the next
instruction. No code wss modified. The conditional branch continues to be present
in
all subsequent computers, while code directly indicating a conditional substitution
is rare. Here is how a conditional transfer would be represented in the code for the
1949 EDSAC (wilkes, 1951, 9):
[A xS yE a]
The first two lines accomplish the comparison through loading [x] (the [A] command)
and subtracting [y (S y; y] is implicitly loaded by the [S] command). [E a] then
transfers control to the code at address [a] if the result is positive and [x ≥
y].
In comparing this with the “First Draft” code, the [A] and [-] or [S] instructions
are both represented as instructions that load and alter a value, but there is a
profound shift in the way that the final branch or substitution instruction is
represented. In the “First Draft” code, the [s] instruction evaluates a value — the result of the [-] instruction — and then places a
value in a location — one of [u or v] — just like any other instruction. Here, a
strict separation is achieved between the processes that are to take place and the
values on which they will operate. Nowhere does the code actually refer to itself,
except in the mind of the programmer, who constructs their
program with the knowledge that, incidentally and not representationally, the code
and data are constructed of the same digital substance and live in the same memory
bank, and one can freely become the other. In contrast, in the EDSAC code, the [E]
instruction evaluates a value — the result of the S
instruction — and then performs an action, a transfer of control, that can only be
understood in relation to the text of the code itself. It directly alters the flow
through the textual representation of the program. That is, in the instruction [E
a,
a] is not a value, but has meaning only through reference to
some other part of the code. Because of this, the operation [E], the conditional
transfer, is unlike every other operation in the EDSAC. Representationally, however,
it takes the same form as [A] or [S]: It is an instruction that performs a process.
The representation of the difference is therefore wholly concentrated in this
address, a. As we will see, it is precisely the address of a transfer instruction
which created the ideological contention within which the first programming languages
were formed.
With the conditional transfer instruction, code fulfills the ideological need to
explicitly represent all of the processes that it engenders. The relationships that
are represented in the code of the Mark I are gone and replaced with a sequential
series of processes. The flow through the text of the program is reflexively
represented in the conditional branch and transfer instructions. The code is stable
and seems representationally clear, according to the needs of its episteme. But as
we
will see, this moment of clarity is already unraveled by the problem of
addressing.[2]
Code and State
Code and state are both invented to resolve the inherent representational instability
of a reflexive, symbolic process. This instability is separated from code, and called
state. Code, then, is constructed to avoid representing this instability as essential
to process. But like any mode of knowledge production which depends on the concepts
of essentiality and inessentiality, code obscures while it clarifies. This supposedly
inessential instability is an inherent part of the calculative process.
While it is not a contradiction simply to represent a dynamic process with a static
text, in the particular semiotics of code, representing a dynamic process means that
somewhere, text dynamically changes. In representation in general, something other
than text, with characteristics other than the characteristics of text, is
nevertheless expressed with text, and the text is only a sign for its reference and
significance. But code comes into being as part of an episteme that demands the
sufficiency of the signifier, that demands that an informational object be constituted by its text, rather than merely signified or referred to by that text. Code, that is,
must not only describe a process, but exist as that process itself. We do not say
that a piece of code “expresses” or “translates” or “interprets” an algorithm; we
say
that code “implements” or “instantiates” an algorithm. While the abstract nature of
the algorithm and its existence apart from any specific textual form is acknowledged
in this expression, the relationship of (concrete) text to (abstract) object is not
thought of as signifier to signified,
but as instance to category. For the
episteme in which code comes into being, the concreteness of the signifier does not
have an arbitrary and external connection to the abstract form of information but
constitutes that information through its own concrete
organizational form. For example, there may be an arbitrary distinction between the
signs [1] and [0], but there is a fourness to the binary figure [1 0 0]. That is,
while the sign 4 has an unstructured and arbitrary relationship to fourness, and
could just have easily been represented as 四, the binary figure [1 0 0] contains a
structured relationship that produces the concept of four through the grammatical
relationship of its elements. These three columns of [1 0 0] show us something about
the relationship between Boolean values within a binary number system, which can then
be mirrored in the physical organization of the electronic process as three parallel
logic lines within a CPU.
When this conception of representation as constitution or instantiation is applied
to
information processing itself, in the form of code, we get a demand for code to
somehow instantiate a dynamic process using the form of a static text. Because the information processed by
code is a text — that is, the information is constituted by its representation — information processing is simply the alteration of one representational form into another. When
code is reflexively taken as its own object — that is, when the result of processing
information is the alteration of the textual flow of the program — this, too, can
only mean the alteration of some representational form. But the alteration in this
reflexive moment thus disturbs the claim that code constitutes its object. On the
one
hand, code represents process, and when the course of calculation changes, the form
of the code ought to undergo corresponding changes. But if code itself shifts, this
shifting is a process, and ought to be represented through some other piece of code. Code always requires something more to represent its
behavior, and so can never really succeed in providing a complete representation of
a
reflexive process.
The way that code addresses this problem is to take the shifting part of this
reflexive representation of its trajectory and to consider it as a part of the
information being processed, rather than as a part of the process itself. This
shifting part is called state. State is the control variables,
the current instruction pointer, the stack of function calls, the layout of the
program in memory, etc. — all the shifting representations of process which allow
the
code to reflexively control the path of its execution. State is supposed to be
incidental, at its limit the mere operation of the machine implementing the code,
and
not the code itself. But this is not possible, as the very reflexivity that gives
rise to the concept of state means that code must itself somehow refer to state
within its text. In order to nevertheless maintain the distinction between code and
state and the inessentiality of state, this reference to state is hidden and buried
within the structure of code. But it does not stay buried, and code is continually
reinvented with new structures that bury references to state in a new way.
It is important to pause here and emphasize that while code, state, and information
participate in a form of representation where the relationship is constitution or
instantiation, this is nevertheless still a form of representation. Neither code nor state are material entities, except by
analogy. State is not the physical world in which the representative calculation
takes place; it is a set of representative forms that change as the calculation
progresses. Code is not a representation of the physical process of the machine but
a
representation of the process of shifting representations. The real process of
calculation in an electronic data processing machine such as a computer consists of
the manipulation of electrical and magnetic fields and currents as they progress
through the hardware, controlled by other electric and magnetic fields and currents,
all of which are only ever translated by analogy to the human-readable
representations with which these fields and currents are thought. Code is a special
kind of representation, not, as some authors have suggested (hayles, 2005), because
there is a causal relationship between representation and process — the process is
wholly carried out by the electronic apparatus without regard to any semantic or
syntactic content — but because code seeks to achieve a total representation of that
process, such that there is a posited equivalence between the representation and that
which it represents. Code is a representation of process that covers over its object
so completely as to almost obviate any need to attend to that
object. Practically, this results in the almost complete
separation of software and hardware concerns. But this almost
continually haunts computer science, and the ghost of the physical world is always
waiting just beyond code.
The Repression of State and the Development of Programming Paradigms
Code and state come into being in the shift from the conditional substitution to the
conditional branch. With the conditional branch, the dependence of code on changing
text is externalized in the address, and the reflexive control of process is reduced
to the reflexive control of the ordering of execution of the instructions. This is
the formation of code in a strict sense. The address, the
means through which this ordering is achieved, is then separated from code (to the
extent that it is usually stored in a register totally separate from the main memory
of the machine). This is the formation of state.
But state only appears in its fully modern form when the state
variable becomes the primary mechanism for the reflexive control of process.
As we will see, because the address was explicitly represented in code as the
destination of the conditional branch, the separation of code and state was
incomplete. In the early history of programming languages, this was resolved through
two transformations. First, the machinic address was replaced with a purely textual
reference. Second, this textual reference was replaced with the increasingly
structured grammar of code itself. But in replacing an arbitrary web of references
with the linear structure of a text, the multidimensional path
of a calculative process must find its representation elsewhere. This elsewhere is
the state variable, a variable whose purpose is to control process, rather than
represent information. And yet, because it is represented as if it were information,
the state variable does not have an epistemic function; it does not adequately
represent its central role in the unfolding of a process.
We can examine these developments through close reading functionally equivalent
conditional branch statements as they are represented in early code (Table 2). Each
code fragment in this table sets some variable [Y] equal to 1 if some value [X] is
less than 5. Otherwise, it continues sequentially.
| EDSAC (Wilkes 1951) | [X in loc. 1, 5 in 2, 1 in 3, Y in 4, code starts at 100]100 A 1 load X from loc. 1101 S 2 load loc. 2 (5) and subtract102 E 106 jump to 106 if ≥ 0103 T 0 clear loaded value104 A 3 load loc. 3 (1)105 T 4 store in loc. 4 (Y)106 … |
| A-2 (Hopper 1955a; Hopper 1955b) | [X in loc. 1, 5 in 2, 1 in 3, Y in 4]0 QTO 002 001 000 is 5 (loc. 2) > X (loc. 1)? 1CN 000 000 002 if so, branch to 21 MV0 003 001 004 set Y (loc. 4) to 1 (loc. 3)2 … |
| Fortran (IBM 1957) | IF (X-5) 10,15,15 branch to 10, 15, or 15 if X-5 is <, =, or > zero, respectively10 Y = 1 set Y equal to 115 … |
| Fortran IV (IBM 1963) | IF (X.LT.5) Y=1 set Y equal to 1 if X < 5 |
| ALGOL 58 (Backus 1960) | if (X < 5) begin Y = 1 end set Y equal to 1 if X < 5, with a block statement |
Table 2.
Table 2: The Development of the Conditional Branch
At the start of the modern code paradigm, the branch address is explicit and
machinic. Hence, the EDSAC code accomplishes the branch with an instruction, [E 106],
whose only parameter, [106], is the address as it appears within the control
structures of the machine. This machine-focused rather than text-focused addressing
has an accidental character tied in with the particular implementation of memory
addressing and the length in memory of a particular segment of code. The address has
little in common with the code it represents. In this way, EDSAC code fails to be
an
abstraction from the particularity of the process and requires constant attention
to
keep its addresses in line with the shifting structure of the code in memory.
Consequently, techniques for creating reusable code through careful attention to this
address form the bulk of the first book on programming (wilkes, 1951) and were
generally a major subject of computer science during the 1950s.[3]
With the A programming languages and Fortran, this address is transformed into a
textual reference. The A series of languages, first developed in 1952 by Grace Hopper
(Hopper 1955a, 23), are centrally motivated by the addressing problem. While A-0 does
little more than rewrite addresses to combine (“compile”) different segments of a
program together, A-2 fully reconceptualizes an address as a location in a text, not
a location in a machine. Each “pseudo-instruction” in the A-2 language is numbered,
beginning with 0 for the first instruction, and any reference to some other place
in
the code uses these instruction numbers, rather than machine addresses (hopper,
1955a) (hopper, 1955b). In our example, the branch statement, [1CN 000 000 002],
refers to its address as [002]: instruction number 2, the third instruction in the
text, which could reside anywhere in the machine memory.
While the A-2 addressing system distances the code from the peculiarities of the
machine, it does little to distance code from state itself. The arbitrary character
of the lexical ordering of the instructions simply replaces the arbitrary memory
layout of the machine. With Fortran, drafted by John Backus and others at IBM in 1954
(sammet, 1969, 143–150) (backus, 1979) (ibm, 1954), this reference to the lexical
order of the instructions is replaced with a symbolic
reference. The [IF] instruction performs a calculation (here [A-5]), and branches
to
three different locations (here [10, 15, and 15]), depending on whether the result
is
less than, equal to, or greater than [0]. These numbers, [10] and [15], correspond
with neither a memory address nor the lexical order of the code, but with labels, arbitrary numbers that are placed before a statement
only when they are needed. The symbolic textual reference thus fully covers over the
shifting instruction address and represents the dependence of code on state through
the lexical reference of one part of the program to another, stitching all the
different ways to progress through the program into a complex web of relations
between different parts of the text.
However, the symbolic reference remains a visible site of the intrusion of state into
code. If each labeled textual fragment could be ordered and structured in the code
according to some unambiguous grammar, then the label could be dispensed with
altogether, leaving only the grammar of code, representing the relationships between
its parts as its own grammatical structure. Fortran IV begins to make this possible
by replacing the three address [IF] statement of the original Fortran with a logical
[IF], which, when its condition is true ([A.LT.5]), proceeds through its conditioned
statement ([B=1]). Here, the relationship between the condition and the piece of code
with which it is related is accomplished solely through grammatical juxtaposition.
But Fortran IV could only place a single expression after a conditional and would
resort to a [GO TO] statement with a symbolic reference for anything longer. Algol
58
solves this with the block statement. While any number of
statements can be placed therein, in Algol anything from [begin] to [end] is treated
as a single statement. A block statement is an organized segment of code which
grammatically functions as if it were a single statement and thus serves to organize
arbitrarily long pieces of code through the grammar of the language.
In the transition from the machinic addresses of the EDSAC to the conditional block
of the ALGOL language, it became possible to write programs defined solely by their
grammatical structure. At first, however, this possibility coexisted with the
symbolic references of earlier languages. ALGOL still provided a [GO TO] statement
to
jump to some other labeled location in the program. The culture of programming had
to
be deliberately transformed to think about code through its textual structure, and
this was the goal of the “structured programming” movement of the later 1960s.
Structured programming is largely remembered through an informal letter to the editor
of Communications of the ACM entitled “Go To Statement
Considered Harmful” (dijkstra, 1969). In this document, Edsger Dijkstra precisely
identifies the epistemic difficulty of code: “to shorten the conceptual gap between
the static program and the dynamic process, to make the correspondence between the
program (spread out in text space) and the process (spread out in time) as trivial
as
possible” (dijkstra, 1968, 147). However, he attributes this difficulty to “our
intellectual powers” generally, and not in their specific historical configuration
(dijkstra, 1968, 147). The GO TO statement, Dijkstra argues, is antithetical to this
correspondence, as it disrupts the connection between the progress through the
program text and the progress through the calculation. The task of structured
programming is to restore that correspondence with clearly structured code, that is,
grammatically structured code.
But in adapting the practice of programming to grammatical structure, the web of
relations between the different parts of a process reappears as a complex
relationship between code and state variables, pieces of data
which work with conditional statements to control the flow of the process through
the
program text. A theorem known as the “structured program theorem” proves that with
the combination of variables and structured code, any algorithm can be implemented
(bohm 1966). Intuitively, a concept like “textual place in the code” can be replaced
with some state variable that somehow indicates which code to run, and subsequently
this variable can be tested with various conditional statements in order to run the
appropriate blocks of code, to skip other blocks of code, et cetera. In this way,
the
path through the code can be arbitrarily altered without explicitly transferring
control to another part of the program. But this reconfigures the reflexive
dependence of code on state as the explicit evolution of a set of state variables within a statically organized text.
This can be clearly seen in one of Dijkstra’s own examples. Dijkstra presents two
programs which test each member of two collections for equality (dijkstra, 1970,
31–32):
CDATA[j:= 0; equal:= true;
while j ≠ N do
begin j:= j + 1; equal:= equal and (X[j] = Y[j]) end
j:= 0; equal:= true;
while j ≠ N and equal do
begin j:= j + 1; equal:= (X[j] = Y[j]) end]
In the first program, we have two variables: [j] and [equal]. We then go through
every value of [j] from [1] to [N], setting [equal] to the logical conjunction of
its
previous value and the result of comparing member number [j] of the two collections
[X] and [Y] ([“equal and (X[1] = Y[1])”]). After the loop is complete, [equal] is
true when the two collections have the same members in the same order, and false
otherwise. The second program modifies this first program to take advantage of a
simple logical fact: If something is false, then the conjunction of that something
and anything at all will also be false. After [equal] first becomes [false], then,
we
know it will stay that way without needing to compare any further. The second
program, accordingly, only keeps looping through values of [j] if [equal] is still
true, and since it only tests X[j] = Y[j] when [equal] is true, we no longer need
the
conjunction with the previous value when we set [equal].
Both versions have unsatisfying elements. The first version provides a relatively
clear representation of what equality is. The logical formula, (X[1] = Y[1]) and
(X[2] = Y[2]) and (X[3] = Y[3]) …, is represented as an accumulation of each
individual test in the variable equal, and the relationship between overall equality
and the equality of each element is represented by looping over every value from [1]
to [N]. However, in representing the formula for equality, it
fails to adequately instantiate the algorithm for equality.
For after we find two unequal elements, we know that the collections aren’t equal,
and we don’t need to test the elements any further. The second program makes use of
this property by putting the variable [equal] into the loop condition. In this second
example, we don’t see the same kind of formula for equality as in the first example.
[equal] doesn’t seem to be an accumulation of individual tests. The conjunction is
missing, and the logical relationship between the equality of different elements of
the collection is no longer explicitly represented. [equal] is still correctly set,
of course, but if we want to know that it will always be correct, we have to follow
the way that the flow of the program is altered by the variable [equal], which in
this second version functions as a state variable whose value
appears to be only incidentally the desired result of the calculation.
This property of the algorithm — that in some cases it need not test every value —
is, regardless of its representative form, the reflexive feedback of a result
(equality) on the flow of the process (whether to stop comparing values). Rather than
representing it with an explicit jump to another place in the code, the structured
programming approach hides this reflexive feedback in the state variable [equal],
but
this hiddenness obscures, rather than clarifies, the function of the algorithm. The
version which ignores this reflexive property represents the formula, what we mean by equality; the version which accounts for this reflexive
property does not. Here, [equal] realizes its proper value through the total
reflexive evolution of the process. But this totality is precisely what is obscured
by the separation of code and state.
While the further developments of programming languages have continued to invent new
ways for containing state (objects, reflection, etc.), largely the conventions of
code were established with structured programming. On the one hand, we have code,
which is supposed to be a clear representation of a process. This process, however,
is dependent on references to a shifting, unstable state. Code is preserved as an
epistemic object by pushing away, containing, and limiting these instances of the
unstable intrusion of state to a set of state variables, such that on the whole code
appears to be comprehensible without recourse to state, except for supposedly
inessential exceptions. But these exceptions are necessary. They are part of the
reflexive nature of any process which uses its own results as a means to control its
progress.
Global Optimization and the End of Code
The epistemic space which the code concept develops is real. Certain aspects of
process become clarified as we apply our understanding — our episteme, with all its
peculiarities — to information processing. But state, and the aspects of code which
depend on it, has relatively few tools to clarify its meaning. What is worse, in
distinguishing code from state, state is posited as an inessential, nonepistemic
object — an object that is unavailable to knowledge — and the general and necessary
connection between code and state is obscured. While in its first 50 years this
epistemic approach was extremely productive in the discipline of computer science,
recently the most important advances in computing all depend in some way on global
optimization algorithms, which are heavily dependent on state. As such, the
constitution of state as a nonepistemic object has become a serious problem.
Global optimization is the process of finding some sort of optimum value, whether
that is some unit of maximum utility or a maximum confidence for some categorization
or evaluation, given a completely arbitrary set of constraints
and zero prior knowledge about the nature of the data. Global
optimization uses various strategies for transforming a solution set such that it
approaches the optimum, however that optimum is defined. In global optimization
algorithms, code defines the way in which this solution set moves towards its
optimum, but it says nothing about the nature of the solution set itself, and little
about the relationship between the resulting solution set and the code which brought
it into being. Consequently, in the code which uses this solution set to react to
new
data, there is often very little understanding of why this set performs as well as
it
does, or under which circumstances it will perform more poorly.
Global optimization forms the core of a large number of research, logistics, and
artificial intelligence algorithms. While it is often studied in different contexts,
machine learning — the process of making choices or predictions about unknown data
based on abstractions from known data, instead of through explicitly given
instructions — is a kind of global optimization problem. These algorithms have
yielded incredibly powerful results in the last 20 years, and we expect them to
become both more powerful and more central to computing.
But global optimization, strictly defined, does not exist. In what is known as the
“No Free Lunch” theorem (wolpert, 1977), it is proven that, when considered over the
set of all possible data sets, no optimization algorithm performs better than any
other. In particular, no optimization algorithm performs better, globally, than
picking values completely at random. What this means is that, in a situation for
which we really had zero prior knowledge about the nature of the problem, on the one
hand there is no reason to suppose any algorithm will produce any kind of adequate
solution set in a reasonable period of time, and on the other hand there is no reason
to suppose that any solution set that currently works will continue to work with new
data.
In practice, global optimization algorithms work because they are not in fact global
but particular. There are patterns of characteristics within the data and constraints
with which they work that, when paired with the patterns of the algorithm, lead to
good solutions. And there are patterns in the solution sets that these algorithms
produce that work because of the way they interact with the algorithm that puts them
into motion. In short, global optimization depends on the exact thing which code
obscures: the inherent relationship between code and state. But global optimization
algorithms, like the rest of code, are structured precisely to forget and to elide
this relationship. The code is still represented as the essential part, and the state
inessential, despite the fact that it is mathematically known that this cannot be
the
case. Thus, there is a proliferation of optimization and machine learning algorithms
that is not matched by our understanding of which algorithm will work better in which
context. Often these algorithms gain credibility through dubious metaphors for
biological processes — “genetic” algorithms, “neural” nets — rather than through a
rigorous analysis of their operation.
As we have seen, this emphasis on code and deemphasis on state is not an accident,
but the historical result of the struggle to achieve an impossible ideological goal
within the material context of information processing technologies: to constitute
changing processes within a fixed representational system. While there is reason to
think that the rise of global optimization might signal a move away from this
episteme, at this juncture that shift has yet to manifest in any fundamental changes
to the code paradigm, which still rigorously separates code from state, if perhaps
less vociferously. Nevertheless, both in computer science and our wider culture there
is an increasing sense of the importance of global optimization, machine learning,
and state.
There are two ways in which code could be adapted to this growing importance. On the
one hand, the paradigm of code could shift in such a way that some new mode of
abstraction could come into being that is able to shed light on the evolution of what
is now called state. Such a shift might bring us an unrecognizable form of code, or
it might bring us wholly different but supplemental modes of conceptualizing the
processing of information. On the other hand, code could simply relinquish its
epistemic function, its ability to represent what we understand about information
processing, and instead we would consign ourselves to the merely empirical knowledge
that some program solved some problem. While this relinquishing might make room for
other modes of knowledge, it just as well might be a political tool to prevent
understanding and effective resistance. Code could become, and to some extent already
is, an ideal unknowable actor, shaping our technopolitical landscape while being
unavailable for effective understanding and resistance.
In this moment (2022), it seems that the most powerful agents within the field of
computing find the obscurity of state to be useful. While hidden code determines much
of our daily experience (pasquale, 2015), nonepistemic code, where the important
actions and connections seem to spontaneously appear from the evolving state of the
program, obscures even the possibility for the responsibility
and choices of its authors. Facebook is “not a publisher” only because the feed
algorithm takes place within a hazily defined state produced from its user data,
rather than in the clearly represented choices of code; and self-driving cars can
only be contemplated because of the difficulty of dissecting state in a
courtroom.
Notes
[1]
Although it is outside the scope of this paper, this episteme itself comes into
being through the general transformation of our social relationship to our
semiotic systems, especially the semiotic system of money. Apart from code, this
episteme is evident in the later development of logical positivism, Chomskian
linguistics, and (with a considerable time lag) the collapse of structuralism
within a Derridean emphasis on process.
[2] While I will be focusing on the address only as it appears
within code and programming languages, addressing itself has a complicated
history. See (dhaliwal, 2022).
[3] As one writer
remarked: “A particular routine may need to be recopied for reasons of legibility
or sequencing of operations. Or perhaps the coder remembers, or finds in checking,
a forgotten instruction. That instruction must then be inserted and all subsequent
addresses modified accordingly” (jones, 1954).
Works Cited
Fuller, Matthew, (2008) Software studies:
A lexicon. Cambridge, MA: The MIT Press.
Haigh, T., Priestley, M., and Rope, C. (2014) “Reconsidering the Stored-Program Concept”, IEEE Annals of the History of Computing 36(1), pp. 4–17.
https://doi.org/10.1109/MAHC.2013.56
Haigh, T., Priestley, M., and Rope, C. (2016) ENIAC in Action: Making and Remaking the Modern Computer.
Cambridge, MA: The MIT Press.
Harvard Computation Laboratory. (1946) A
manual of operation for the Automatic Sequence Controlled Calculator.
Cambridge, MA: Harvard University Press.
Hayles, K.N. (2005) My mother was a
computer: Digital subjects and literary texts. Chicago, IL: The University
of Chicago Press.
Heide, L. (2009) Punched-card systems and
the early information explosion, 1880–1945. Baltimore, MD: The Johns
Hopkins University Press.
Hopper, G.M. (1955a) “Automatic
Programming: The A 2 Compiler System — Part 1.”
Computers and Automation 4(9 September), pp.
25–29.
Hopper, G.M. (1955b) “Automatic
programming: The A 2 Compiler System — Part 2,”
Computers and Automation 4(10 October), pp.
15–27.
IBM. (1954) Preliminary report:
Specifications for the IBM Mathematical FORmula TRANslating System,
FORTRAN. IBM Programming Research Group, Applied Science Division.
IBM. (1956) Functional wiring
principles. New York, NY: International Business Machines.
IBM. (1957) Programmer’s primer for
FORTRAN Automatic Coding System for the IBM 705 Data Processing System.
n.p.: International Business Machines Corporation.
IBM. (1963) IBM 7090/7094 programming
systems: Fortran IV Language. n.p.: International Business Machines
Corporation.
Jones, J.L. 1954. “A survey of automatic
coding techniques for digital computers.” Master’s Thesis, Massachusetts
Institute of Technology.
Kemper, Carrie, writer. (2019) Silicon Valley. Season 6,
episode 6, “RussFest.” Created by Mike Judge, John
Altschuler, and Dave Krinsky. Directed by Matt Ross. Aired Dec 1, 2019. HBO Home
Video.
Marino, M. (2014) “Field report for
critical code studies, 2014.”
Computational Culture 4 (November). http://computationalculture.net/field-report-for-critical-code-studies-2014%e2%80%a8/.
Marino, M. (2020) Critical code
studies. Cambridge, MA: The MIT Press.
Montfort, N. and Bogost, I. (2009) Racing
the beam: The Atari Video Computer System. Cambridge, MA: The MIT
Press.
Pasquale, F. (2015) The black box society:
The secret algorithms that control money and information. Cambridge, MA:
Harvard University Press.
Sammet, J.E. (1969) Programming languages:
History and fundamentals. Englewood Cliffs, NJ: Prentice Hall.
Von Neumann, J. (1993 [1945]) “First Draft
of a Report on the EDVAC”, IEEE Annals of the History
of Computing 15(4). https://doi.org/10.1109/85.238389
Wilkes, M.V., Wheeler, D.J., and Gill, S. (1951) The Preparation of programs for an electronic digital
computer. Reading, MA: Addison-Wesley.
Wolpert, D.H. and Macready, W.G. (1997) “No
free lunch theorems for optimization”, IEEE
Transactions on Evolutionary Computation 1(1 April) pp. 67–82. https://doi.org/10.1109/4235.585893
Backus 1960 Backus, J. (1960) “The syntax and semantics
of the proposed International Algebraic Language of the Zurich ACM-GAMM
Conference”, in Information Processing: Proceedings of
the 1st International Conference on Information Processing, UNESCO, Paris 15–20
June 1959, pp. 125–131. Paris: UNESCO.
Backus 1979 Backus, J. (1979) “The history of Fortran
I, II, and III”, ACM SIGPLAN Notices 13(8
August), pp. 165–180. https://doi.org/10.1145/800025.1198345
Böhm and Guiseppe 1966 Böhm, C., and Giuseppe J. (1966) “Flow
diagrams, Turing Machines, and languages with only two formation rules”,
Communications of the ACM 9(2), pp. 366–371. https://doi.org/10.1145/355592.36564
Campbell-Kelly et al. 2014 Campbell-Kelly, M., Aspray, W., Ensmenger, N., and Yost, J.R.
(2014) Computer: A history of the information machine.
Boulder, CO: Westview Press.
Clippinger 1948 Clippinger, R. F. (1948) A logical coding
system applied to the ENIAC. Report No. 673. Aberdeen Proving Ground, MD:
Ballistic Research Laboratories. Available at: https://apps.dtic.mil/sti/pdfs/ADB205179.pdf.
Dhaliwal 2022 Dhaliwal, R. (2022) “On addressability, or
what even is computing?”
Critical Inquiry 49(1 Autumn). https://www.doi.org/10.1086/721167
Dijkstra 1968 Dijkstra, E.W. 1968. “Go to statement
considered harmful”, Communications of the ACM
11(3 March), pp. 147–148. https://doi.org/10.1145/362929.36294
Dijkstra_1970 Dijkstra, E.W. (1970) Notes on structured
programming. T.H. Report 70-WSK-03. The Netherlands: Technological
University Eindhoven.
Eckert and Mauchly_1945 Eckert, J.P. and Mauchly, J.W. (1945) Automatic high-speed computing: A progress report on the EDVAC. Moore
School of Electrical Engineering. Pittsburgh, PA.



