This presentation is part of the 2017 3D Digital Documentation Summit.
Photogrammetric gigapixel imaging of monumental wall paintings using visible, infrared and ultraviolet radiation.
Speaker 1: Hello. It’s great to follow on from that presentation from [Che 00:00:04], obviously I’ve used a lot of their resources online in doing the research that I carried out. Before I get started, do you want me to introduce myself a bit more, I’m a wall painting conservator, so I come at this technology from a slightly different point of view from a lot of the presentations that we’ve seen so far.
I’m a freelance wall painting conservator working in London with the Courtauld Institute of Art where they work mainly on international projects that I work with, but the two projects I’m going to discuss today are at the Royal Gallery in the Palace of Westminster which is this image here, the painting isn’t blue, this is a fluorescence or luminescence image and also the painted hall in the old Royal Naval College at Greenwich Maritime. Greenwich in London.
So for me, what’s been interesting over the past couple of years as I’ve investigated 3D documentation techniques for the application of wall painting conservation is the ever growing availability of the hardware and the software and the greater user-friendly nature of capturing 3D data that means that nonspecialists in the field such as conservators can now start to use these techniques themselves where the technology is available to them.
So the two projects that I’m going to talk about, the kind of primary output from the projects is not necessarily 3D data but as Che was talking about it’s the 2D orthographic images that are the primary output focus for the projects. And what I’ve been trying to investigate is combining this technology, photogrammetry with multi-spectral imaging and seeing if it’s possible to combine these technologies to create better documentation of wall paintings using different light sources and different sorts of imaging techniques and seeing if they can be overlapped and worked on a much greater resolution and to a greater scale than they have done in the past.
Usually with multi-spectral imaging image sets, the size of the sample area is usually quite small, maybe a meter squared, whereas in this case I’ve been applying it to wall paintings that are more like 10 m² or bigger.
So these are the two sites that I’ve been working at. On the left ear as you look at the screen, this is the painted Hall at Greenwich and then on the right it’s the Royal Gallery just outside the House of Lords in the Palace of Westminster, and so the paintings in the Palace of Westminster are about 11 m long and the paintings on the ceiling at Greenwich are about three times as long as that.
So the documentation was one component in the preliminary stages of these wall painting conservation projects and the time constraints necessitated that the high-resolution documentation is captured quite rapidly particularly at Westminster, the room is very high-profile room that’s constantly in use. Essentially, it’s an open plan office for the Lord’s and is also used for state functions and so getting access to the room for photography is very different world and so when access is available capturing the highest resolution, the best data that you can is imperative, but just for future reference the windows above the paintings become important in terms of the conservation project in the use of the output images that I’ve created.
In the royal gallery of the Palace of Westminster the two image sets that were captured were visible and UV luminescence images and I’ll go into a little bit later on what UV luminescence images are but the two paintings are on plaster. They’re technique called water glass paintings which is quite a specific technique used in northern Europe to replicate the effect of true fresco technique which is where the painting is executed on wet plaster rather than dry plaster and I think there are a couple of examples of these, this technique in America but the paintings depict two of the most important battles in British military history.
On the right there you have the Battle of Waterloo and on the left, Battle of Trafalgar. These were painted in the mid-19th century by Daniel Maclise. So the project scope was to capture the invisible and UV luminescence images and the resultant images were then used both as a record of the present condition of the painted surface for our conservation project which is ongoing and also to investigate the physical history of the paintings with reference to original and non-original added materials such as coatings and areas of re-painting. And it’s also being used as a monitoring tool during work on the stained glass windows above the paintings.
This is the old Royal Naval College at Greenwich, the building with these paintings is to the right there and it’s slightly behind this building in the foreground. It’s painted by James Thornhill in the early 18th century and the building is by Christopher Wren and the building was built in the late 17th century.
So as I’ve said before, the project aims are more or less the same for both the sites but the royal gallery essentially the difference is that visible and UV luminescence imaging was done whereas in the painted Hall visible and IR reflected imaging was done and this is because it was not possible to do the luminescence imagings in the painted whole because all the imaging was done from the floor here and to do the luminescence imaging you have to get within a couple of meters of the paintings themselves due to logistical and technical reasons to do with ambient light and the amount of radiation we can produce from the flashes.
Just a little bit about the documentation of wall paintings and why photogrammetry is particularly well-suited to wall paintings. As we can see, wall paintings are often monumental in scale. They have a three dimensionality, they’re often heterogeneous in terms of condition, original technology and also added materials and therefore an accurate and effective documentation of wall paintings is a technical and logistical challenge.
They vary specially, how do we capture this information, how can we capture all these different surfaces, for example, in this room here, more or less all the different surfaces whether in the room are painted. So being able to capture this three-dimensional nature of wall paintings is particularly important. A large-scale recording of this complexity is paramount for effective conservation development in implementation. It can inform our understanding of materials and it can help us form hypothesis regarding deterioration phenomena and it can also help us develop treatment proposals, it can help us understand access requirements, where do we need to build scaffolding if we’re going to conserve these paintings.
It’s an extremely expensive undertaking to build scaffolding in this space and so do we need it everywhere or do we only need it in a localized space? So high resolution photography and 3D digital documentation tools can help us investigate prior to the scaffolding or where we might need to access.
Here’s another example, this is a Byzantine church in Cyprus, I’ll just show you again, the three-dimensional nature of wall paintings means that conventional photography falls short of documenting them accurately. Distortions due to single points of view, areas that are not visible, domes and arches, recessed niches, objects in front of them, how can we use the technology available to us now to accurately record these wall paintings in the same way that the building itself is recorded because wall paintings and the building structures are entirely interlinked and so the best form of documentation of the wall paintings is often the same as which can be applied to the building.
And the challenge of documenting wall paintings is not only that it’s also the location where they are. Often wall paintings have a public social-religious function meaning that access to them is usually restricted or difficult to negotiate. This is the House of Lords in the room next to the one that I was doing the project in and it itself has wall paintings high up on the ceiling. The use of the building renders access to these paintings extremely difficult so we need to use technology to help us both record them and investigate their condition to help develop treatment proposals.
And then on the left here, we have this is the royal gallery again, but after the scaffolding has gone up in the royal gallery and the wall painting is actually now behind that plastic protective sheeting so once that scaffolding’s up photography of the wall paintings is more or less, extremely difficult in a meaningful sense.
So the provision of platforms and scaffolding is a common and costly necessity for imaging and other forms of recording whilst it improves access to the painting it also limits flexibility, field-of-view and position relative to the subject. We also have obstructions often to do with the devotional practices that in some instances are sacred and can’t be moved. How can we use technology to negotiate these things?
Moving on to Greenwich where we have also again, the spatial complexity of wall paintings. These are the paintings that are in the dome of the vestibule and the very nature of this domed shape to the wall painting means that conventional photography can’t really accurately capture the information. Also, from the ground you can actually see all the wall paintings because of the dome shape and therefore capturing them in three dimensions facilitates the general public from being able to see these paintings. We can also visualize them in orthographic projections to help us understand the relationship to the architectural space.
This is a series of ceiling panels in Rajput Palace in Rajasthan from the 18 century where we’ve got severe problems with delamination of the wall paintings away from the ceiling itself and so being able to correlate that condition to the failure in the structure of the building is particularly important to understanding what the causes of the failure in the wall paintings may have been.
And these are techniques and ways of visualizing the wall paintings that have not really been applied, have only recently … The technology for the conservator to create these images has only really been very recently available.
We also have the material complexity of wall paintings both in terms of the pigments that are used to create them and also applications such as varnishing, gilding, that’s one aspect of the material complexity. The other is the condition which they’re in. The condition of wall paintings is often subject to aspects of the structure itself or other external forces and the fact that wall paintings are the interface between the building and the external environment means that their condition is often particularly bad.
So how can we record all the different things going on within this image. Actually, hidden beneath this image is a very beautiful Baroque wall painting but how can we begin to understand all the different overlapping conditions that will help us restore them?
The common way to do this is using graphic documentation to map different condition phenomena and also different aspects of original technology, this is an example from a cave site in northern India where photogrammetry has been used to map, to model the cave, it’s a very small cave on conglomerate rock and photogrammetry has been used to create a model and that therefore we can use an orthographic image exported out of the model to more accurately represent the wall painting itself and then also more accurately map condition phenomena and original technology.
So as I mentioned before, I’ve applied photogrammetry not only to standard photography, that is reflected visible light but also to luminescence imaging and I’ve combined these techniques to spatially record the materials and conditions of the wall paintings.
For more information about multi-spectral imaging and the techniques used in this case you can refer to the British Museum project, the Charisma Project which is available on the British Museum website that provides guidelines and software about how to create different types of reflected images and also different types of luminescence images.
So the methodology applied has been a flash-based system which is particularly good for documenting wall paintings where often the photography is done off scaffolding or in tight spaces. In the past lamps have been used but the advantages to using a flash-based system, the fast shutter speed and speed of capture that allows us to capture more images more quickly. The technology is relatively low cost and available off-the-shelf. The size of the areas that can be sampled in a single image is limited by physical considerations such as architectural space, available lenses as well as the power of the radiation source.
And also as we can use the fast shutter speeds because of the use of flash it allows us to cancel out the ambient lighting in the room which in the case of the Royal gallery with the stained glass windows it’s not possible to control and daylight often in the past it would have been done at night but because we can use very fast shutter speeds we can actually do it during the day and not all have to come in in the middle of the night which is advantageous.
So this is a diagram of the difference between reflected and luminescence imaging. Here we have visible reflected imaging and infrared reflected imaging where the lights or the radiation shown onto the wall painting or whatever it is you’re photographing is the same as that which the camera detects, whereas in luminescence imaging for example, and the standard UV-induced luminescence imaging, you shine UV light onto the wall painting and what you capture is visible light and its separate from the UV light and therefore any visible light that the camera detects is either created by the wall painting itself or the painting itself or its from another source in the room if you haven’t calculated in ambient conditions or maybe there is some parasitic light in your radiation source but we can include reference samples within the images that helps us understand what is actual luminescence and what is just parasitic light.
So this is a very nice example of where it can be practically used. This is a painting in Rajasthan of Krishna and on the left here we have the normal visible image and then the UV luminescence image on the right and the difference between the two images is the color you see in the image on the right is all produced by the painting materials itself off of the excitation with UV light and what we’re seeing is this bright yellow fluorescence in the area where we’ve got a yellow pigment and what that’s telling us about the painting is that it’s probably an organic pigment and it is an organic pigment which has implications for the conservation treatment that we would use.
You might not actually be able to see in detail but if you zoomed in on this image you would also see tiny specks of orange fluorescence which is the lapis lazuli used in the blue which is an impurity of natural ultramarine and that’s fluorescing and so we can use this to noninvasively identify the use of lapis.
And another technique, so we’ve got a visible image, an infrared reflected image and we can align these two images to create an infrared false color image where we’ve taken the red channel of the infrared image and the red and green channel of the visible image and then putting them together in a composite image known as a false color image that can be used against reference standards again, to noninvasively identify pigments and also to visualize differences in the painting and how the painting is responding to the different forms of light that can give us information about maybe it’s history or its condition.
So how does this relate to more general survey techniques in terms of wall paintings. We need to think about what is needed in terms of wall painting conservation. What is needed is techniques to capture the scale and complexity of the subject and also the color and surface information because obviously it’s a painting so we’re interested in the color but also the surface information about things like presence of coatings, varnishes, whether the surface is deteriorating or whether it’s in good condition.
So in the past rectified photography has often been used but it has certain limitations in comparison to photogrammetry which is becoming much more increasingly available where we can capture high quality RGB images that can be used to create both 3D models as we’ve seen but also these high resolution orthographic images that allow us to virtually inspect the wall painting from a distance.
And then we have laser scanning which for the purposes of my use is not something that’s been available to me because it’s very expensive but then also it doesn’t have the advantages that the photogrammetry may have for our interests in terms of color and surface information.
So if we look at this handy diagram from the SEPA conference, we can see that wall paintings fit somewhere in the 1 to 10 meter range and the complexity of the object is such that close range photogrammetry or laser scanning is a nice technique to choose.
So this is an example of the orthographic images, ortho-mosaic that were created. This is the painted Hall in Greenwich, what we’re interested in as wall painting conservators is being able to move in to the highest resolution possible through the images that we capture so we can begin to understand things about why the painting is maybe deteriorating or why it needs conservation. In this case it’s to do with the presence of a varnish that is altering and obscuring the painting and so being able to map from the floor and look at the painting in this kind of detail really helps us prior to the erection of scaffolding to understand what may be going on with the painting and also target certain areas for further investigation.
So in terms of photogrammetry and capture methods for what we might need to do, essentially wall paintings fall into all of these categories. We might have a façade like we do at Westminster, or an entirely painted interior space as we saw in the Byzantine church or we may have a column or other architectural features where we will need to adapt the capture methodology to get the information that we require.
So these recent advances in structure for motion software packages can be coupled with the fluorescence and infrared imaging as we’ll see in a moment.
So as we’ve seen in the last presentation so I don’t need to go in too much detail about capture techniques and the best methods for doing this, but we have an overlapping unordered image sequence that we can use to produce a 3D model where each point in your subject needs to appear and so many photographs, nine it seems, and images can be captured using handheld captures and off-camera flashes which is particularly good for relatively fast capture of images.
So this is an example of the sequence of images captured at Greenwich. So really the advantage for the wall painting conservator is that there’s no specialized equipment beyond photographic equipment that we already have available. It’s the software that is the more expensive and more unusual component to this for conservatism.
So the structure for motion software packaging merges these different photographs and calculates the position of the photograph creating this 3D mesh but really as a conservator in terms of for paintings what we’re interested in is the color information.
And the final resolution is a result of those original images. So the higher quality original images that we get the better the information that we might finally have in our final ortho-mosaic or model. And so this is the Battle of Waterloo and so it’s Wellington meeting Blücher just after the battle, it’s a fantastic painting but conservation is mainly focused on presentation issues to do with the painting. And so we’re really interested in aspects of the physical history of the painting to do with the application of coatings and varnishes and areas of repainting that we may want to reduce to improve the legibility of the paintings because essentially it you go into this space there is a beautifully dulled room and there are these very dark wall paintings and so the focus of the conservation project is how to improve their legibility.
And one of the proposals is better lighting in the space because as a result of the investigation into the surface of the painting it’s been decided, or it’s in the process of deciding how much should be done physically to the object and how much should be done changing lighting in the room and other conservation choices that we have.
So for the photography, for the visible and luminescence imaging a Nikon D8 10 was used, whereas for the IR images you have to adapt a standard camera and take out its internal IR filter and replace it with a quartz filter.
And so the radiation sources, they’re the same for all the different types of imaging except for the visible imaging we have different filters on the camera and then for the IR imaging we change the filter so we just see the infrared light and then for the UV luminescence imaging we put some filters on the flashes and then also put some different filters on the camera itself so we can selectively choose what type of light is both shown on the painting and that which the camera captures.
So the color is accounted for by using the both color checker except the UV luminescence images because white balancing doesn’t apply in the standard ways because essentially we’re photographing a light source of an unknown color and so it’s set to a neutral cloudy setting and then we also include spectralons which allow us to calculate how much reflected light is in the image as opposed to how much luminescence is in the image. And so up to 700 images were captured for the paintings and they were processed using my computer which had limitations so where I essentially crashed my computer, I could break the paintings down where necessary and for all the processing of the models the quality settings were set to medium.
So this is an example of the capture technique in Westminster, my scaffolding wasn’t quite as movable as they had for the Diego Riviera but I had a team helping me there so essentially I was about 2 meters away from the painting, that’s determined by the distance you have to be from the painting for the UV luminescence imaging and I worked my way along the painting capturing many, many images whilst the room is being used by the Lord’s and that was particularly important to get permission to do the imaging it was to not shut the room. Because if the room gets shut people start shouting, essentially.
So this is an example of the imaging with the flashes, with including the reference material to help us calculate what’s happening in the images and so for the Westminster paintings there are approximately 50 m² each, and we have a lack of control over the ambient lighting meaning that we need to use the flash by system and so this is an example of one of the images and you can see here, these are our spectralons, so the whitest image, which is a bit blue is a reflectant material, so all that is actually reflectant and not luminescence so we know that there is some reflected light in our image and we can use the British Museum guides to help us subtract that out so we can begin to look at the images and understand what’s going on with them in terms of the information that they reveal to us.
So this is an example of one of the images that was captured covering the entire space of the wall painting and revealed to us is aspects of essentially the paintings have been heavily repainted and this is creating a muddy effect on them and they’ve also had many different applications of varnishes and coatings, some of which have been attempted to remove creating a patchy, uneven effect to the paintings.
So being able to map this across the whole surface of the painting is an extremely good tool when it comes to treating the paintings and potentially improving their legibility. So again we’ve panned across the surface capturing our image sets in both visible light and UV induced visible luminescence.
So we can see for example, there is a slight red fluorescence to his red jacket which seems to have been repainted and also the same, similar pigments have been used in areas of his face, which is his ears and on his cheeks and we can also begin to compare differences between areas of the repainting.
So these are the two paintings, Trafalgar at the bottom with the death of Nelson, Nelson’s lying there dying in the middle and Waterloo above it. So these are the visible reflected images and these are the UV induced physical luminescence images. So we can look at the whole of these paintings in UV light and to a very high resolution based on this type of resolution. Something that’s not been possible before as I said, usually you sample one area or you might move around depending on how much time that you have, so we can begin to look at effects on a much larger scale than was previously possible. And we can begin to see things like these cracks which are important for the conservation work that’s ongoing at the moment and also we can map out the areas of repainting and coatings.
And so this is again some more details from the two images. So this is an image, oh sorry, this is a video that we can hopefully will load relatively quickly. So this is just loaded into light room which has some file size limitations so these are actually not the full-size images but we can use them to compare them and digitally inspect the wall paintings off site and begin to plan our conservation campaign off-site and not necessarily in front of the paintings. Also, not necessarily moving around the paintings with lots of UV light both irradiating us and the paintings for too long.
So we can see in areas such as the background here, these areas have been heavily focused upon in previous conservation or restoration attempts. Also the blue tunic we can see has been repainted, all the faces are muddy and dark due to the application of these phases of repainting. We can also see differences in the repainting. Some of them are extremely fine, others like this are not fine but I think if we go down we might actually see some. [inaudible 00:33:50]
Because we are not sure … Immediately after these paintings were created they began to fail due to this experimental technique called water glass that was used to create them so actually, the Daniel Maclise, the painter went back very soon after painting these paintings to … [inaudible 00:34:14] he went back very soon after painting the paintings and repainted some of them himself in a different material to how they were originally painted.
So being able to separate out where Daniel Maclise repainted them and where someone else much later on came and did these washes over the surface to disguise the deterioration that was happening is a very important to treating the paintings.
So we also have, as I mentioned before, these cracks that were revealed in UV light and these cracks were known about before but it seems like they’re more extensive than they were originally thought, it’s likely they’re the result of the bomb that was dropped during World War II and the way that these paintings are constructed is they are on a lath and plaster support separated from the main body of the wall with a void and then plaster, dry plaster and then the paint on them.
And so what’s happening at the moment is there is work on the windows above, the far right image is an image of the work that’s happening on the windows that are above the paintings, so quite invasive work with mechanical drills to replace some of these stone elements do to water ingress and that’s having a big effect on these paintings because you’re gonna have to imagine they’re quite a thin material that’s quite flexible with cracks running through them.
So at the moment the concern is during this work high vibrations are being produced and it may cause these cracks to fail further and cause loss to the painting. So at the moment we’re structurally monitoring, oh sorry, we’re monitoring the vibrations that are being transmitted through the painting and using these high resolution images to assess if there is any change in the wall paintings, particularly focused on these areas of cracks where you do have a lot of delamination and so we’ve used 3D information that was captured prior to this work to compare the delamination as it is now with the delamination that we can see in the model.
So in the case of Greenwich, the imaging that was undertaken was infrared imaging and visible imaging from the floor a long way away from the paintings, unfortunately I had to use a zoom lens and not a prime lens in this case but it’s allowed us to both use the images for publicizing the conservation project that’s ongoing in Greenwich, but also as a tool to inform our conservation decisions when it comes to treatment and so this is the visible image of the paintings and the infrared image and when we start look in more detail at the two images we can compare them side-by-side and bring out aspects that we would want to focus on in terms of conservation and focus on in terms of conservation for such as the differences that we see in the robes here are probably areas of repainting or selective use of non-original coatings which are now failing.
And this is an example of the images combined and at the end is the infrared false color image that shows up much greater in terms of looking at a larger scale using the infrared false color image allows us to quickly identify areas that may be of concern for the conservation of the wall paintings.
And I will just skip through these and acknowledge a few people who have been involved in the project.
Caroline Babington at the Houses of Parliament, also Paine and Stewart, a Conservation company in London, and the Courtauld Institute of Art, and Sreekumar Menom from India. Thank you.
The challenges of documenting wall paintings are severe; limited access, large scale, immovable heterogeneous condition and technology, complex geometry. However, digital imaging can now overcome many of these challenges, allowing us to digitally interrogate the surface of wall paintings in ways previously limited to small sections of the whole painting or museum pieces such as panel and canvas paintings. The increased availability of high-quality imaging hardware and software puts in the hands of the conservator digital tools to accurately document and investigate wall paintings, aiding the conservator in the understanding of original technology, non-original materials, physical history and condition.
This paper will discuss the innovative use of high resolution photogrammetric imaging with visible, infrared (IR) and ultraviolet (UV) radiation. A structure-from motion software package (Agisoft’s Photoscan) was used to process large sets of images (>200 individual images) to create orthomosaics of the paintings under reflected visible and IR (X-Nite, 830nm cutoff at 50%) radiation as well as using UV-induced visible fluorescence (365nm). The resultant orthomosaics can also be processed into IR-false colour images to aid in pigment identification, areas of repainting etc.
This technique has been employed in the preliminary stages of two conservation projects, Daniel Maclise’s monumental wall paintings in the Palace of Westminster and Sir James Thornton’s masterpiece in the Old Royal Naval College in London, UK, and form the case studies for this paper. Photogrammetic imaging has allowed the paintings to be visualized in new ways overcoming the constraints of imaging wall paintings in their architectural context. The combination of the three types of imaging, visible, IR and UV builds up a comprehensive record of how the paintings respond to radiation across the spectrum. This means we can map areas of repainting, varnishes, areas of deterioration in unprecedented detail and visualize this without perspectival distortion; usually a challenge with the documentation of wall paintings. Furthermore, it helps inform the conservation decision making process as well as being an archival record of the paintings prior to conservation.
The methodology of these techniques was specially designed to overcome the challenges of accurately, effectively and efficiently documenting, insitu, large scale wall paintings in high resolution. The use of a flash-based system for the UV fluorescence and IR reflected imaging facilitated imaging from scaffolding with short exposure times (<60th second), during daylight hours and whilst visitors and staff were also using the space. This flexibility in how the images are captured means the process can be incorporated into the conservation workflow where scaffolding is usually required for access.
These techniques can be built into conservation workflows at minimal cost whilst providing high quality qualitative and quantitative data and archival records and are an exciting development in how wall paintings are imaged.
Samuel Whittaker undertook an MA in the Conservation of Wall Painting at The Courtauld Institute of Art. His research dissertation focused on non-invasive investigations of fluorescent properties of natural organic colorants. He then took up a graduate internship at the Getty Conservation Institute in Los Angeles. During this internship he researched the application of 3D-imaging techniques to map deterioration phenomena and quantify rates of change through image analysis. He has been a conservator on projects in the UK and on Courtauld projects in India, Bhutan and Cyprus and has supervised MA conservation students at Bundi in India, Tamzhing in Bhutan and Vardzia in Georgia.