Some months ago now, I was prompted by Microsoft Start to read the piece at reference 1. By some route or other this led me to one of Oliver Sacks’ books, which led me in turn to a paper about the configuration of strokes used to make the letters of alphabets. Some of this was noticed at reference 2. I then turned to reference 3, which takes a look at the sort of configurations that can be made from a small number of straight-line strokes.
Reference 3 contains a catalogue of the connected configurations which can be made from up to three such strokes, organised by the number of junctions. This is snapped in the figure above. Part of the argument there is that the distribution of these configurations in the letters of alphabets – or perhaps in written texts – is similar to that to be found in nature. To this end, configurations were sampled from suitable photographs. I was not terribly convinced, but I was intrigued, I thought I would try some of this for myself.
But before getting onto that, I consider how such strokes might come to be found in digital images.
Edges and strokes
In the analysis of digital images, a key concept is that of the edge. Identifying the edges is a way into the digital muddle of a raw image, a large rectangular array of pixels. And by extension, the array of pixels somehow coded onto the human retina. In both cases, two dimensional representations of a three dimensional world – allowing that, sometimes, the important part of that three dimensional world might, in effect, be two dimensional, perhaps the page of a printed book.
There are some hints as to how this might be done at reference 4, more than thirty years old now, but they still seem entirely plausible.
We suppose first that each pixel is coded for colour in some way. The we have an edge when we have a linear, one dimensional discontinuity in that colour in our two dimensional array. Where by linear, we do not necessarily mean a straight line, but we do mean something reasonably smooth and well behaved.
We suppose second that each pixel is coded for depth in some way, a supposition made plausible by our binocular vision. Then we also have an edge when we have a linear discontinuity in that depth.
Some edges will be the result of colour discontinuity, some of depth discontinuity, some of both. Reference 4 is all about how edges might be labelled and how this facilitates reconstruction of the three dimensional world from the two dimensional image, but for present purposes all edges are the same and they are all called strokes, bounded by a suitable circle, often but not always drawn around a junction of edges.
Two such are shown in the snap above, one of each kind. Colour alone, above and left; colour and depth below and right. In both cases the edge, the boundary between the two zones, is very clean, very like a Euclidean line, with length but without area. There is no boundary zone, at least not at this level of magnification.
I associate to the boundary zones of reference 5, snapped above. Any line that the experimenter might see here is a confection of his eyes and brain. One might also add that these eyes and brain can detect remarkably small differences when samples are placed side by side in this way. Much harder when they are a little way apart, not to say impossible when there is visual noise in between.
Here things are not so clear. The brain knows that there is an edge, at least very probably, but the eye cannot see it. What to do?
Here, however, in the snap of ivy leaves above, we do have boundary zones. Boundary zones which are not just an artefact of the camera and computer, they can be seen, at least at times, with the naked eye, out in the field.
Most leaves do not have edges. But they can be seen in other thick, flat leaves, such as holly, laurel and some ornamental magnolias, and no doubt other leaves of this sort. Most deciduous leaves are not of this sort; flat but not thick.
I suspect such leaves as being of fairly unform thickness, not tapering to thin margins and of having fairly blunt margins, rounded but blunt – broad enough to catch some light and with some trick of that light then giving us the white lines. F in the left hand panel of the snap above rather than C – this snap of sections of blades of grasses being lifted from reference 6. The scale bars are 200μm, a fifth of a millimetre.
While the more uniformly rounded margins in the right hand panel come from reference 7 and the large tropical hardwood trees of the genus Parashorea. The scale bars here are also 200μm.
In what follows, I shall treat such boundary zones as edges or strokes.
Perhaps more tricky, do we treat the bars circled left as the two stroke configuration 03 (T)? Or do we treat it as something more complicated, a confection of four narrow rectangles, two light and two dark. I suppose, as ever, the answer is that it all depends. In what follows, I shall score it as 03 (T). And the crossing to its left as 04 (X).
Strokes and configurations
Using these strokes, bounding circles and the catalogue of configurations we opened with, we now look at three more or less natural scenes. Are there configurations to be found?
To keep things simple, we restrict ourselves configurations involving just the one junction and three or fewer strokes. This keeps the number of configurations we are looking for down and means that every junction defines a configuration – which means in turn that we do not have to bother with selecting suitable junction pairs or junction triples. Or configurations which are part of other configurations – a statistical complication if nothing else, and one which does occur in the real world.
The tulip tree flower
A 07 (Ψ) configuration formed by the veins of a leaf.
A 03 (T) configuration formed by a petal occluding the leaf behind. An easy way to make a T.
Two 02 (L) configurations, one formed by a corner in a petal, the other by a corner in a leaf. We overlook the rounding of the junction: one has to allow a certain amount of ‘snap to grid’ in these matters. We note that the three-dimensional reality underlying the yellow L might well be quite complicated. That underlying the leaf L probably not.
A rather ‘impure’ 04 (X) configuration formed by the petiole of one leaf crossing the edge of another.
A 06 (K) configuration formed by a chance alignment of petals. An artefact of the point of view as much as of the flower itself, of the subject as much as of the object.
A rather faint 05 (Y) configuration formed by a twig in the background.
The logs
A rather ‘impure’ 04 (X) configuration left and a rather ‘impure’ 03 (T) right.
Another not very clean 03 (T) configuration lower left, a rather better one upper right. This last, the edge of one object partially occluding the edge of another. Complicated in this case by the bright edge being between one face and the invisible but shadowed and adjacent face of the same object, rather than between the object and background.
Another not very clean 02 (L) configuration upper right. Two 02 (L)’s which are arguably 03 (T)’s middle right and lower left. But L is what strikes one in the first instance.
Bottom right we have a 05 (Y) formed by the chance alignment of a corner with an edge. While the other three are the much commoner view of a corner of a vaguely brick like object.
The one on the left could easily have been a four stroke X, excluded on our rules. My impression is that this sort of X is commoner than the two stroke sort, 04 (X).
A 06 (K) configuration. Which might arguably be a four stroke version, which would be excluded.
Another rather weak 06 (K) configuration.
The ivy leaves
A rather weak 05 (Y) configuration.
Lots of 02 (L) configurations formed by the points of leaves.
A 05 (Y) configuration formed by a chance alignment.
A 03 (T) configuration formed by the branching of a flower part. A T which is not formed by two distinct objects, as are most of the others here.
A 04 (X) configuration formed by the crossing of two flower stalks.
And I close this section with a junction which is excluded on any likely limit on the number of strokes. A three dimensional feature of the natural world which is not readily captured by one of our configurations.
Comments and conclusions
We have kept things simple, restricting ourselves to configurations involving just the one junction and three or fewer strokes. Configurations defined by the way that their strokes interact with each other, rather than by their details. Generalising from the facts on the ground to the topology, as it were.
Only a proportion of the configurations that we have identified are what might be called clean and we have allowed a fair amount of ‘snap to grid’, tidying up the raw image, to get our configurations. There are also a good number of cases where more than one interpretation is possible. We are doing rather more than a straightforward feed-forward contrast edge detecting algorithm might do, bringing some top-down intelligence into play.
That said, one can extract these configurations from natural scenes, as is claimed at reference 3, although I found just the one example of 07 (Ψ) and I did not find examples of 08 (man) or 09 (asterisk).
And then there are the configurations with more than three stokes or more than one junction, which do seem to crop up too. I suspect, for example, that asterisks made up of six (or more) strokes are more common than those made of three.
But it all seems a bit forced. That this is not a particularly productive way to get into, to parse natural scenes. That said, it would be curious if the distribution of configurations so found did turn out to be similar to that found in writing, as is argued at reference 3. A matter to which I shall return.
It is probably not relevant that I am not usually conscious of these configurations when I look about me: I would expect the parsing of scenes on the retina to be taken care of by the unconscious, certainly most of the time, when there is no problem or ambiguity with the scene in question. One might then, for example, consciously use 03 (T) junctions to sort out foreground and background objects. Probably not 15 (TL) or 28 (drum).
One might add that the lead author, while still busy and active at reference 7, does not appear to have pursued this particular line of inquiry, despite this paper appearing second on the citation list at reference 8.
PS: in the margins I have learned about general flowering, a feature of the Dipterocarpaceae family, of which the Parashorea are part. General flowering being synchronised flowering which happens from time to time, as opposed to year after year. Apparently, a spectacular business when it happens. Plus, some panda important bamboos do it, which is bad for the pandas. The mallows, mentioned recently at reference 12, are a sibling family.
References
Reference 1: Optical illusion reveals key brain rule that governs consciousness – Emily Cooke, Live Science, Microsoft Start – 2024.
Reference 2: https://psmv5.blogspot.com/2024/08/building-letters.html.
Reference 3: The Structures of Letters and Symbols throughout Human History Are Selected to Match Those Found in Objects in Natural Scenes - Mark A. Changizi, Qiong Zhang, Hao Ye, Shinsuke Shimojo – 2006.
Reference 4: Interpreting Line Drawings of Curved Objects – Jitendra Malik – 1987.
Reference 5: https://psmv5.blogspot.com/2022/10/an-instrument.html.
Reference 6: Variation in leaf anatomy within the Elytrigia intermedia – E. ×mucronata – E. repens (Poaceae) hybrid complex – Ladislava Paštová – 2017.
Reference 7: Leaf anatomical and micromorphological characters of some Malaysian parashorea (dipterocarpaceae) – T Noraini, DF Cutler – 2012.
Reference 8: https://www.changizi.com/.
Reference 9: https://scholar.google.com/citations?user=XPXdsFkAAAAJ.
Reference 10: https://en.wikipedia.org/wiki/Mark_Changizi.
Reference 11: https://swartzcenter.caltech.edu/. His one-time home.
Reference 12: https://psmv5.blogspot.com/2024/08/back-to-library.html.
No comments:
Post a Comment