Telegraphy, the 19th century version of network communications, introduced the transmission of messages and news in coded form using pulses, and soon came to be used to transmit images, first as facsimiles using electro-chemical techniques and later as television and the wire-photo. Many of the coding systems employed a version of the binary code which is at the heart of all computing systems. The pulses used in the telegraph could also produce the synchronisation necessary to make coherent two-dimensional images using the facsimile, television and the raster-monitor display.
There was also great interest in the harmonic behaviours of sound waves and many ingenious devices were developed for their display. These included Chladni figures (made by shaking a light dusting of sand on a vibrating plate), Lissajous figures (produced from the combination of simple sound waves) and various devices for recording sounds with a moving quill (or pen) onto a roll of paper.
As the understanding of electricity developed (and techniques for producing a vacuum in a glass tube improved) it was discovered that cathode rays (electrons) would stream from a piece of heated metal in an evacuated tube and would cause shadows of objects placed in the stream to be projected onto the end of the tube. With certain chemicals painted onto the end of the tube (phosphorous being useful here) the stream of cathode rays would cause the “phosphor” to light up. It was also found that magnets placed around the tube would cause the cathode ray stream to be deflected from its normal path. Thereby the cathode ray tube could be made to display waveforms (and thus the cathode ray oscilloscope was born, with its descendants the radar display and the vector monitor) and, when combined with the synchronising techniques of the facsimile, television and the video and computer monitor we know today were not so far away.
When electronic computers were developed it became necessary to store the programs that would run them so that the computer could access the instructions and the data in its own time. That is, memory was required. Originally a modification of the cathode ray tube allowed the storage of data bits on its screen in the form of a grid (a raster). Because the data was stored in the phosphor of the screen in this so-called, “Williams-Kilburn Tube” it could be seen and thus the computer memory also became a display and this is the basis of the bitmapped screen memory.
But let’s go back to the discovery that data (numbers) could be represented in a coding system in the first place.
The prime function of all computing is in the manipulation of sets of numbers; whether they represent actual numbers, characters in words, or pixels in an image. The decimal numbers and zero came to us via the Arabs who had preserved the knowledge of the Greeks as well as conveying the knowledge of mathematics that originated in India.
The Binary number system, Leibniz, and the I Ching
But the form of numbers used in modern computing is the binary system discovered in Europe by the German philosopher and mathematician Gottfried Willhelm Leibniz (1646 – 1716). He had been searching for a universal language which would justify his Christian theological views and allow him to prove that his God had created the world out of nothing. Following on the work of the 13th century Spanish Christian mystic
But the form of numbers used in modern computing is the binary system discovered in Europe by the German philosopher and mathematician Gottfried Willhelm Leibniz (1646 – 1716). He had been searching for a universal language which would justify his Christian theological views and allow him to prove that his God had created the world out of nothing. Following on the work of the 13th century Spanish Christian mystic Ramon Lull (ca. 1232-1316), whose Ars Magna1 purported to represent all the characteristics of the Christian God as a set of propositions, the subjects and predicates of which could be combined to represent the properties of all aspects of the world, Leibniz realised that the characteristics and properties in question could be represented as numbers and thus as algebraic variables. He believed that all things were created ex nihilo (from nothing, from the void) and he equated the void with zero (0) or nothing and his God with one (1), since, for him, God represented the unity of the cosmos. This led Leibniz to argue that number lay at the basis of all creation2 and that the creation of the world could be represented as arising from zero and one:
“Omnibus ex nihilo ducendis sufficeit unum”
One [1] is sufficient to draw all things from the void [0].).
Joachim Bouvet, a Christian (Jesuit) missionary in China at the time (c.1700) 3 had noticed that the hexagrams of the YiJing (I Ching), being made up of whole and broken lines, corresponded to Leibniz’ binary number system. Within the YiJing the yin (the broken line) and the yang (the unbroken line) represent all binary oppositions, thus manifesting the world through the combination of their properties. Bouvet sent Leibniz a copy of Shao Yong’s commentaries on the YiJing which led Leibniz to propose that the Chinese had, in their ancient past, not only a significant lost mathematical science based on the binary number system but that they also had some sort of Christian basis to their spiritual beliefs. 4 A view the Chinese refused to accept. More importantly it led Leibniz to point out that the binary numbers could represent all possible numbers using just two symbols and, given that, he further established that all philosophical propositions and thus all philosophical arguments could be represented in symbols which could then be mapped onto binary numbers allowing them to be tested by mathematical analysis. Later (in the 19th century) this discovery formed the basis for Boole’s mathematical analysis of logic and led him to develop the Boolean logic of AND, OR and NOT. 5
Leibniz announced his discovery of the Binary numbers and their Chinese association in a communication (“Explanation of the Binary Arithmetic”) in the Memoires de l’Academie Royale des Science, of 1703. 6
The binary numbers demonstrate the progression of all numbers as a periodic geometric progression in which each subsequent pair of numbers, even and odd, are represented by an extra column of 0 and 1, which he demonstrated with his table of the first 32 numbers [Fig.1]. They also gave Leibniz a sound basis for a Number Theory:
“But in lieu of the progression of tens in tens, I have employed for several years the simplest progression of all, which goes by two in twos, having found that it offers a basis for the perfection of the science of Numbers. In this manner I do not use characters other than 0 and 1, and then on reaching two, I recommence. This is why two is written as 10, and two times two or four as 100, and two times four or eight as 1000, and two times eight or sixteen as 10000, and so on.” 7
Although Leibniz found that he could do useful theoretical work with the binary system he commented that he wouldn’t adopt it for ordinary decimal calculation:
“For beside what one is already accustomed to, one has no need to learn what one already has learned by heart, in this manner the practice of tens is the most abridged (succinct, the epitome), and the numbers there are less (not as) long.” 8
Nevertheless, he did design a binary calculator but it was not implemented 9 although he subsequently designed a decimal calculator that was the first to perform multiplication and division as well as addition and subtraction. The multiplication mechanism used a specially stepped drum-gear which could be set to control the number of times a number was added to achieve multiplication. 10
Automata
The technology of gears, levers and cams formed the basis of both clockworks, calculators and, later in the 18th century, automata. To a large extent automata demonstrate the development of the idea that one could make a machine carry out a sequence of different moves supposing it possessed the necessary mechanism. This could be done by embedding the sequence in the machine as a set of rotating disks having gears and cams which drive the mechanism’s levers and might trigger another disk to commence its actions at certain points in its the sequence. This is very much the basis of programming and, in automata, often various sequences of actions might take place in parallel, or branch depending on a certain condition arising. Essentially automatic (or program controlled) sequential machines, they used various types of cams with small knobs (cleats) on them that, being mounted on a common axis, formed a cylinder which rotated under clockwork or hydraulic power and moved (via levers) the various elements (limbs, head, eyes, etc) of the automaton, or struck tuned elements for producing sounds in music boxes.
Generally once the cylinder of cams was installed the machine was fixed in terms of its possible behaviours and any change of program was extremely difficult.
The great French automata makers Vaucanson and the Jaquet-Droz brothers produced remarkable machines based on clockwork driven cams. 11 Vaucanson’s most celebrated automaton was a duck which was said to be able to flap its wings and eat and digest food, producing pellets of waste at the end of the process. 12 The Jaquet-Droz brothers produced several humanoid automata including musicians 13 and a boy who could write.14 Chapuis and Droz describe it thus:
“The automaton is sitting on a Louis XV style stool before a small mahogany table. He holds a quill pen in his right hand, while the left hand rests upon his writing table. The head is mobile thus the eyes turn in all directions.” 15
In many ways these mechanical wonders might be regarded as the origins of artificial intelligence or at least of robotics.16 More importantly for us here, Vaucanson also used his cylinder of cams in early attempts to automate the weaving looms of Lyon, France, which were used to produce the expensive brocaded silks used in fineries for the ruling classes of Europe. These patterned cloths probably sit at the roots of all modern (computer) graphic production.
The Jacquard Loom
Although patterned elements in textiles and carpet weaving have a long and visually interesting history, it can be argued that the first programmable graphics system is seen in the products of 18th Century French weaving loom technology. There are a variety of ways in which one can define a computer graphic depending on which aspects of computing one chooses to emphasise. If one considers that the essential element of computing is programmability then versions of the weaving loom that could be sequentially controlled using mechanisms similar to, or derived from, the technology of automata, especially those that used a string of cards that enabled programmed control of the loom, may well be the first programmable graphics machines. Thus the automated loom sits at the roots of all contemporary bit-mapped graphics, and provides an important development of two key concepts; the first in computing:
- that a sequence of operations of a machine could be pre-programmed and stored in a medium which could be placed into and removed from the machine so that the sequence could be altered easily. This is the concept of a programmable machine, and the second in graphics:
- that an image of considerable detail could be worked into a grid (or raster) made up of vertical and horizontal lines having different colours at each point that a horizontal line (the weft) crosses a vertical line (the warp). This is the basis for the bit-mapped or raster graphics now ubiquitous in computing.
The weaving of decorative silks involved a vast number of repeated actions which required two operators. Selected warps of the loom had to be raised by a “drawboy” according to which colour was to appear at any point in the weaving and then the coloured thread (the “weft”) was led through the warps by the master weaver. 17 This was time consuming and required two workers to operate the loom, making woven silk textiles extremely expensive. A series of attempts to alter this situation were made over the 18th century. In 1725, Basile Bouchon invented a means for controlling the drawing up of the warps using a roll of paper perforated with holes matching the pattern to be woven into the cloth. The paper roll was stepped through the reading mechanism using sprocket holes in the edge of the paper much as was used in the lineprinters of early modern computing. In 1728 Falçon replaced this roll of paper by a system of punched cards pressed onto a block of wooden needles that lifted the required warp threads [Fig 2]. In 1745 Vaucanson developed a system in which the paper roll was replaced with a perforated cylinder perhaps to get a more compact system [Figs 3a & 3b]. This was basically a modified form of the cam technology that he had been using in many of his automata. 18
By the end of the 18th century Joseph-Marie Jacquard had taken Falçon’s idea of punched cards in a linked chain and produced a reliable mechanism that could be attached to the loom so that the cards could control the new steam powered looms. The cards are drawn onto a box cylinder surface one at a time. A set of needles are pushed against each card and those that penetrate the holes in the cards engage hooks which are attached to the warps by strings thus drawing up the required warps, leaving the weft threads to run under the raised warps and appear over those warps not drawn up. Each card [Fig 4] represents one colour of the thread to be shuttled through the warps in one line of the cloth as the warps are lifted or not, under control of the holes in the cards, thus determining the colour placed at each point in the image.19 The punched cards are strung together in a chain [Fig 5], and if the pattern needs to be changed any number of the cards can be removed and replaced with other cards with the replacement pattern punched into them 20. If symmetry in the pattern was required the machine could be “backed” (operated in reverse) using the same set of cards in reverse order, or if a pattern was to be repeated the machine could be backed up to the start of the pattern cards and the sequence repeated. These two procedures became very useful to Charles Babbage in his designs for programming his Analytical Engine.
Jacquard first showed his loom in the Paris Exhibition of 180121 and his developments were completed by 1804. However the period of development from Bouchon and Falçon up to Jacquard‘s launching of this technology had massive political consequences. Essentially the Jacquard mechanism put half the weavers out of a job, riots ensued and many of the machines were smashed. It took until 1815 before the Jacquard became well established in France, but this was perhaps more to do with a fashion for plain cloths at the time.22
In his Passages from the Life of a Philosopher, Charles Babbage describes how the Jacquard cards were made for producing images:
“It is a known fact that the Jacquard loom is capable of weaving any design which the imagination of man may conceive. It is also the constant practice for skilled artists to be employed by manufacturers in designing patterns. These patterns are then sent to a peculiar artist, who, by means of a certain machine, punches holes in a set of pasteboard cards in such a manner that when those cards are placed in a Jacquard loom, it will then weave upon its produce the exact pattern designed by the artist.” 23
This process permitted the production not only of intricate repeatable patterns on ribbons, its primary use, but the execution of very elaborate graphical images of the precision of an engraving [Fig 6]. At the headquarters of the company producing the Jacquard loom in Lyon, France, a special set of 24,000 cards24 was made which could reproduce a portrait of Jacquard standing in his studio holding several of his punched cards. 25 [Fig 7] This portrait was woven on demand for special guests. In 1840 Babbage travelled to Turin in Italy to attend a meeting with L.F. Menabrea 26 and other mathematicians. His journey took him via Lyon to visit the factory where the Jacquard portraits were woven to see the loom upon which they were produced. He was allowed to purchase one of the portraits to present to the Queen on his audience with King Charles Albert of Sardinia whom he was also to visit. 27 As the portraits were only made on order he was then able to closely watch its production on the loom. Babbage also possessed one of these portraits of Jacquard, which was often mistaken for an engraving. 28
The Jacquard technology used by the artisans of weaving then flowed directly into the history of computing. Babbage recognised that the chain of cards used for “programming” the weaving loom could also be used to program any other repetitive process and this would apply to the repetitive actions of his Analytical Engine as readily as it would to a loom. He designed the Analytical Engine to extend the capabilities of his Difference Engines into a flexibly programmed machine which could handle any number of different algorithms at any stage in the sequence of its operations.
He realised that he would need
“a set of operation cards … strung together, which contain the series of operations in the order in which they occur. [And a]nother set of cards … strung together to call in the variables into the mill, [in] the order in which they are required to be acted upon.”29
These two chains of punched cards, one to handle the sequence of operations and the other to supply the numbers (the variables), represent in modern terms the program and the data that the program would act upon, while the mill is the central processing unit (CPU). Babbage then recognised that these card sequences could be broken up into chunks so that a process could proceed so far and then on the occurrence of some condition could branch into a new process and then on the satisfaction of some subsequent condition could return to the original sequence or depart into yet another new branch.30 His system of supplying the program and the data by Jacquard cards evolved into the Hollerith card which was ultimately taken up by IBM and formed one of two primary means for entering programs into the early generations of computers (the other being the punched tape which we will meet in the discussion of telegraphy below). Meanwhile the Jacquard system is still in use in weaving today, though the warps lifting mechanism is now directly computer controlled.31
There is no indication in Babbage’s writings or in Lovelace’s discussion of the Analytical Engine in her Notes by the Translator 32 that either of them considered that it might be used to produce images of any sort. Nevertheless Lovelace did recognise that the engine could operate upon objects other than numbers if those objects could be represented to the machine in the appropriate manner. Thus she foreshadows computer music:
“Supposing, for instance, that the fundamental relations of pitched sounds in the science of harmony and of musical composition were susceptible of such expression and adaptations, the engine might compose elaborate and scientific pieces of music of any degree of complexity or extent.” 33
However Babbage did envisage that the Engine would produce stereotype plates for the printing of tables of numbers as its output.
For Lovelace the Jacquard cards enabled the Analytical Engine to be a (programmable) machine that “weaves algebraical patterns just as the Jacquard-loom weaves flowers and leaves.” 34 and, but for the lack of a suitable output device (e.g., an interface to the Jacquard loom), she may well have conceived Mathematical Art as well as Scientific Music, for the Engine was certainly capable of doing the maths. 35 This is of course “worthless speculation” and we will leave the matter there. However, by the mid 19th century there were thousands of Jacquard looms operating in France and England. They were used mainly for decorative cloths for fashion and furnishings but the capacity to create images with them led to a major industry of making sentimental pictures, book marks and commemorative cards (often known as Stevengraphs 36) that were woven for public sale as part of the many Great Exhibitions in England (beginning in 1851), Europe from 1869 and in the USA. [Fig.8]
The Jacquard loom was introduced into Japan in 1877 37 [Fig.9] and the technique continues to have wide use in the weaving of textiles for ceremonial use in Buddhist and Shinto temples, as well as for the sashes (Obi) used with a kimono. Most of these patterned ceremonial textiles from Japan use gold wire as a thread producing a very bright glistening result [Fig.10]. The type of carpet known as an Axminster carpet is also produced with a Jacquard controlled mechanism. An example of this method of carpet making may be seen in the Wool Museum in Geelong, Victoria.
The Jacquard-controlled weaving process has direct relation to the production of images from digital computers. The cards carry information about which warps to lift as a row of holes along the length of the card. There are several ways in which the grey tone or colour to be used is encoded in the pattern of holes depending on the kind of textile being woven, but usually a separate row of cards is used for each colour to be used in a line of weft. The multiple rows of holes in a card then enable a wider spread of “needles” across the loom. In the contemporary language of computer imaging the warps equate to the columns and the wefts are the rows of pixels on the monitor. Thus, a woven image is the logical equivalent of a bit-mapped video display. However, this story does not reappear until the latter half of the 20th Century.
———————————————————-
FOOTNOTES
1 Lull, Ramon. (1517) Ars magna generalis et ultima… ed. Bernard de la Vinheta. Lyons: Jacob Marechal for Simon Vincent, 5 May 1517. 3rd Edition. The most accessible modern reference would be Martin Gardner’s Logic Machines and Diagrams, Chapter 1. [Gardner, 1958]
2 Swetz, Frank J (2003) “Leibniz, the Yijing, and the religious conversion of the Chinese” Mathematics Magazine; vol. 76, no.4, (Oct 2003) pg. 281.
3 https://en.wikipedia.org/wiki/Joachim_Bouvet. Leibniz-Bouvet Correspondence, Translation and Annotations by Alan Berkowitz and Daniel J. Cook. http://leibniz-bouvet.swarthmore.edu/
4 ibid, p.284; and Ryan, James A. (1996) “Leibniz’ Binary System and Shao Yong’s Yi Jing” Philosophy East & West, vol.46, no.1, (January 1996), pp.59-90.
5 Boole, George (1854) An Investigation of the Laws of Thought, on which are founded the Mathematical Theories of Logic and Probabilities, London: Walton and Maberly.
6 Leibniz, G.W. (1703) “Explication de l’arithmetique binaire, avec des remarques sur son utilite, et sur ce qu’elle donne le sens des annciennes figures Chinoises de Fohy”, Memoires de l’Academie Royale des Science, vol. 3 (1703), 85-89.
7 Leibniz, G.W. (1703) “Explication de l’arithmetique binaire …”, in Gerhardt, C.I. (1962) G.W. Leibniz Mathematische Schriften, vol.7, pp.223-227, Hildesheim: Georg Olms Verlagsbuchhandlung, p.224, (transl. Stephen Jones).
8 ibid. p225. [transl: Stephen Jones]
9 Leibniz, G.W. (1679) De Progressione Dyadia, dated March 15, 1679, mentioned by Swetz (opcit, ref. 2).
10 Williams, Michael R. (1990) “Early Calculation” in Aspray, William (ed) (1990) Computing Before Computers, Ames, Iowa: Iowa State University Press. , pp.42-49.
11 Some of these may be seen in the Centre National des Arts et Metiers in Paris, though the duck is not there. (Its whereabouts are unknown.) Some of the Jaquet-Droz automata are in the Musée de Neuchâtel, Switzerland.
12 Brewster, David (1868) Letters on Natural Magic, London: William Tegg, pp.320-321 and Chapuis, Alfred and Droz, Edmond (1949) Les Automates: Figures Artificielles D’Hommes et D’Animaux: Histoire et Technique. Neuchatel, Switzerland: Editions du Griffon., pp.239-243.
13 Chapuis and Droz, 1949, op cit, pp.287-291.
14 Chapuis and Droz, 1949, op cit, pp.301-304.
15 Chapuis and Droz, 1949, op cit, pp.301. (translation: Stephen Jones)
16 One should also remember the Golem made from the clay of the Prague river by one Rabbi Loew in Prague, Czechoslovakia, in the 16th century.
17 Kennedy, Frank. “Joseph-Marie Jacquard” . Kennedy cites Chauvy, Bernard. Joseph-Maire Jacquard and the Weaving Revolution. and Decker, Rick and Hirschfield, Stuart. The Analytical Engine. PWS Publishing Company: Hamilton College, 1998.
18 Rouille, Philippe. (1993) “Trous de memoire (Memory gaps)” La Revue, Feb. 1993, no 2, p.34-41. Musée des Arts et Metiers.
19 Rothstein, Natalie (1977) “The Introduction of the Jacquard Loom to Great Britain” in Gervers, Veronika (ed) Studies in Textile History. In Memory of Harold B. Burnham. Royal Ontario Museum, Toronto, Canada. p.282.
20 Rouille, 1993, op cit, p.166. English translation by schindler@cnam.fr, 3 Oct. 1994.
21 Rothstein, 1977, op cit, p.281.
22 Rothstein, 1977, op cit, p.282. It was, after all, not that long since the Revolution and the wearing of anything ostentatious could well have been quite dangerous.
23 Babbage, Charles (1864) Passages from the Life of a Philosopher, London: Longman, Green, Longman, Roberts and Green, p.117. The Passages…is the nearest thing we get to an autobiography from Babbage.
24 Lovelace, Ada (1842) “Notes by the Translator to: ‘Sketch of the Analytical Engine invented by Charles Babbage, Esq.’ by L.F. Menabrea, in Bibliotheque Universelle de Geneve, No.82. Oct.1842”. in Taylor’s Scientific Memoirs, Vol.III, pp.666-731: Note F “There is in existence a beautiful woven portrait of Jacquard, in the fabrication of which 24,000 cards were required.” Also in Bowden, B.V. (ed.) Faster Than Thought, A Symposium on Digital Computing Machines, New York, London, Toronto, Pitman Publishing Corporation, 1953, p.395.
25 See also plate 15, facing p.113 in Hyman, Anthony. (1982) Charles Babbage – Pioneer of the Computer. Oxford: Oxford University Press.
26 Menabrea later became the Prime Minister of Italy. It was this meeting that produced the Menabrea paper describing Babbage’s Analytical Engine for the Bibliotheque Unverselle de Geneve (October 1842). Ada Lovelace translated the paper into English and wrote her extensive “Notes by the Translator” for the translation when it was published in Taylor’s Scientific Memoirs. See Bowden, B.V. (ed.) (1953) Faster Than Thought. New York: Pitman, Appendix 1. The translation and the Notes demonstrate her very considerable mathematical understanding and she offered several insights into the future of analytical machines, including matters of programming, computer music and artificial intelligence. See Toole, Betty Alexandra. (1992) Ada, The Enchantress of Numbers, Mill Valley, California: Strawberry Press, pp. 257-8.
27 Moseley, Maboth (1964) Irascible Genius. A Life of Charles Babbage, Inventor. London: Hutchinson, p.142.
28 Babbage, 1864, op cit, p.169. Babbage rather gives the impression of being what one might now think of as a republican, however he seems to have quite taken to Albert, the Prince Consort. He relates a story of the Prince’s visit. Babbage showed him the portrait of Jacquard hanging in his drawing room. “When we had arrived in front of the portrait, I pointed it out as the object to which I solicited the Prince’s attention. “Oh! that engraving?” remarked the Duke of Wellington. “No!” said Prince Albert to the Duke; “it is not an engraving.” I felt for a moment very great surprise; but this was changed into a much more agreeable feeling, when the Prince instantly added, “I have seen it before.” I felt at once that the Prince was a “good man and true,” and I resolved that I would not confine myself to the rigid rules of etiquette, but that I would help him with all my heart in whatever line his inquiries might be directed.”
29 Babbage, 1864, op cit, p.118.
30 Babbage provides a detailed description of the Analytical Engine in Chapter VIII, pp.112-141 of his Passages from the Life of a Philosopher. [Babbage, 1864]
31 Janice Lourie developed the computer controlled Jacquard at IBM in the late 1960s and early 1970s. See Lourie, Janice and Bonin, A.M. (1968) “Computer-controlled textile designing and weaving.” Information Processing 68, Proceedings of IFIP Congress 1968, pp.884-891. Edinburgh, 5-10 August, 1968; and Lourie, Janice (1973) Textile Graphics/Computer Aided, New York: Fairchild Publications.
32 Lovelace, 1842, op cit.
33 Lovelace, 1842, op cit, Note A; reprinted in Bowden, 1953, p.365.
34 Lovelace, 1842, Note A, her emphasis; reprinted in Bowden, 1953, p.368.
35 eg, see. Lovelace, 1842, op cit, Note E; reprinted in Bowden, 1953, pp.386-395.
36 Godden, Geoffrey A. (1971) Stevengraphs and other Victorian silk pictures, Rutherford [N.J.] : Fairleigh Dickinson University Press.
37 Quarritch Rare Books catalogue 1301, Design and Innovation, notes to item 182, 2002.
Return to Contents