This presentation is part of the 2017 3D Digital Documentation Summit.
3D Digital Documentation; From a field of specialization to an interdisciplinary point of interaction
Speaker 1: Good morning, everyone. First of all, thank you Kirk Cordel and Jason Turf for having me here. Like Kirk said, we met in 2012 and I was looking forward to seeing the topics that were laid out this year, and I was really excited to be a part of it. I think I’m looking forward to all the lectures that will be happening throughout the day, and tomorrow.
I know Kirk already gave an introduction to me, so I’m going to very quickly tell you. I took a 26 hour flight to come here, so apologies if anything doesn’t work out. And, if there are any glitches, I’m using also a MAC presentation, so when the power point, there are changes, I made it into a PDF. So, I will be switching between the presentation for some video parts of it.
So, basically as Kirk introduced, we started a lot of work in DHARMA since 2006. Before this I had been working in Germany, at the technical university Dresden With 3D technology. So, I have at least 12 years of experience with the field, and have been … Well, actually more than that, 15 years of experience in the field, and 12 years with Notre Dame doing the work that I have been doing. So, coming to the work of 3D documentation and technology, I’ve got to know both sides of the spectrum.
At Notre Dame we do traditional work. And, we do a lot of hand drafting. So, from there to the latest technology in the field is a huge belt to cover. And, we try and find a way to interweave the two.
So, the presentation today is going to talk about three different projects. The ones Kirk mentioned, the Taj Mahal, the Roman forum, and the Belvedere Vatican. And, they will be expressing exactly how we go about the process, and also encompassing how the field has transformed over time in the techniques and methods that have been used.
I will conclude with a simple issue of questions that still remain open for the future. And, we can have some questions after that from the audience. For people who have been in the field more longer than I have, you definitely know also the traditional techniques that have been used. But measure drawing is not a new method at all. Actually, it was used by many in the field since antiquity and beyond. There have been ways in which people have documented techniques of drawing of plans and off sites throughout the globe.
One of them that I picked for today was the Temple of Apollo at Didyma. Didyma, which is 540 B.C. when it was initially constructed, you know it was one of the typical Greek temples that we know of, but when professor Axelrod, when he found the drawings, he saw these. I don’t know how many of you are aware of these. Has anybody seen this before, just so I can get a sense of gauge. This is a drawing that was found directly on the surface of the temple of Apollo. And when they saw this they realized this was a measure that was used to construct the temple.
This is a scaled drawing that was proportionately one eighth of the whole temple and it uses, on your right you will see the order is compressed and shown in a very compact form. So the anthesis on top and the one below are expressed in just a compressed drawing. So this is a very advanced technique of measure drawing that was visible and understood already in antiquity by people that were working on site. While we think that Renaissance is responsible for drawing techniques, there’s already understanding of it known to people in antiquity.
There is of course, also the marble plan of Rome that we all are aware of. This marble plan, gigantic in scale, found at the temple of Peace, is another example of how the city of Rome was mapped onto a marble surface and it was visible and understood and recognized as a scaled information regarding the city and how it was evolving throughout time.
Over centuries, later on you can see the Renaissance right and in this time you take examples that were found from antiquity and find new ways of interpretation for it. So the understanding of forced perspective, the understanding of how do you scale and survey a site, becomes much more systematic and organized. And these are then found in treatises like the Alberti Treatise that we see that explains us exactly how to document a site in a thorough manner.
Interestingly though, 1500 years have gone by but you see still very simple techniques that are being used for scaling and surveying and the drawing part is the one that people spend the most time on.
So on site there is a difference in the time ratio that is spent in composition and creation of drawings. People are working with single points and they are working on creating a survey map that will be worked out for drawings that take at least four to five months to construct. In case of the Roman forum for example, it took at least five years to build the full detail drawing that was put into use.
This is a very pixelated drawing but it was very difficult to find first psy-arch scanner that was ever put together. I had a picture of it some time ago when Ben showed it to me, but now I lost it so I picked it up from online. This box, how many of you are aware of … Yeah. The first laser scanner that was put together by Ben Caterra, you can see it was a bulky piece. You cannot imagine taking it to a site anytime. It was extremely difficult to navigate and had a very low point to frequency understanding of how much information he can grasp.
Today, we are way further. 15 years have gone and we can go more than 30,000 points per second, even more. We have new advances in technology. The scanner is becoming much more compact. It is navigable. I remember, 2006 I had like a scanner which had these huge batteries. Does anybody remember this? Oh my god, that was … We were sweating through the forum. We did not know how this going to work.
Today we take these lithium batteries that tare with us. And even further we take the latest drone from robotics which is able to, you can take the scanner along with it, and you can attach the data together. There’s a whole different vast understanding of documentation that as gone from 15 years, 1500 years. The technology, the evolution, is amazing. And how far we have come, how quickly we have come across with it, is just mind blowing.
The Google cars for example are using the scanner technology. Also they are using them to do completely different things. They are able to take a car and move it through the site and avoid accidents. But privately, they are mapping all of California. As the cars are moving around they are collecting millions and millions of points of data. Some of them are being thrown away, but mostly it’s allowing you to create a map of the whole state to a profound degree than it was ever possible before.
So what are we doing with all that we have understood in the last years? We are trying to map or read the understanding from the 1500 years to that of the last 15 years. So in DHARMA, which is a long name, the Digital Historic Architectural Research and Material Analysis. And trust me, I did not come up with this name. People believe it’s Dharma so it’s associated to it, but no, it isn’t. It was my American students who came up with this name, so I have nothing to do with it.
We basically look at world heritage sites and we look at understanding how you can preserve them for our prosperity. We work in different ways towards collecting the knowledge and making it into a common pool of resource that would be visible and useful for scholars, different scholars in the field, and then create new tools and techniques to enhance the understanding of virtual dimension.
What does that mean? Lots of gibberish that you spoke. First of all, it means we work with the heritage site itself for structural and surveys material analysis. We work with 3D scanning and field study, and we use the giapan technology. Anybody who uses Giapan in the room? Yes, so there are a couple about that. I’ll explain if people have questions. And then we merge the data together to facilitate integrated understanding of it. And then beyond that, we take accurate measurements and use that for research interpretation and also for actual conservation work.
So the work of DHARMA is divided into two parts. Primarily we work on one side with research and the other with practical conservation. So for the practical conservation part we work with various teams that are responsible for the conservation on site. So for example, at the forum we worked with the Soprintendenza, the ministry of conservation there. We work at Taj Mahal with archeological survey of India, and at the Vatican with the architectural team of the museums.
For research, that’s an agenda we set up for ourselves. So it’s a research team that is doing particularly new advancement in understanding what we know about that site or about actually the field of technology that we are working with.
So these two then at some point interact with each other and we get new results based on the work that comes together. These are never seen separately, but for clarity purpose I have separated the two.
Just a quick list of our team members, this is a student who was at the forum. The temples of Vespasian, temple of Vespasian, anybody who knows the Roman forum, in Rome, the temple of Vespasian, it’s very difficult to get up there. We’re not permitted to be there at all. And these are students who’ve actually found new findings about the temple beyond scholars who’ve already been in place. So they’re incredibly excited and happy to be a part of this program. So we get a huge contribution from students and that’s how we run basically.
Most of the work is done through a team of scholars and a team of students that work very hard from five in the morning till sometimes nine in the evening. And are completely burned out sometimes. In the sun, in heavy heat, it’s summer research that we do.
We have partnerships with various organizations. We have a four year memorandum of understanding with UNESCO. And this is critical. You’ll say, “Why is she talking about this?” But this is important to be able to get access to some of the major sites where it’s incredibly difficult as a researcher to come in. So the collaboration of research and practical work allows us also, they see the practical component of it, they are actually able to conserve, and we are able to do research with it, so it’s a win/win.
We also never charge. So we don’t, nobody pays us any money for any of the research we are doing or for the efforts that we are doing on site. So it’s very much based on learning more than anything else. And we have industrial partners that help us get our equipment and so on, and so forth.
Okay, so just for the research part now and this is going to be very, very quick because I’m going to go directly to the three sites we are working on. We work on mapping two technologies together and now we have integrated that with many more. But the scanning technology and the mapping field is completely separated right now. What happens is you go on site, you do 3D scanning. And you are also doing photography. Then you come back and use different softwares to be able to map the 3D photographs on top of the 3D scan. You either create a mesh or a model, and then depending on the speed and the capacity of your computer, the amount of data collected needs to be reduced to be able to get a good model.
Many times, with very high capacity work it becomes incredibly difficult to process large amounts of information. So we see statues and we see materials that have come out in wonderful 3D sculpture forms, model forms. But when it comes to buildings, it becomes really difficult to capture them in 3D. So some of the work we are doing is to illuminate the 3D mesh surface part of the mapping and just go from cloud directly to mapping of the photographic technology.
This is done in house. This is a PhD student who doesn’t get any sunlight … Who is trying to find a way to merge onsite mapping of the scanning and gigapanning to happen in the same field. So briefly what we do is we keep the scanner in the same location as our photographic mapping takes place. We’ve actually created a device that allows the two to operate simultaneously and then we go from point to point to capture the same amount of information, one in 3D and the other in 2D.
What that allows us to do is then directly map the surface information together and you can see there are gaps that you get based on the amount of scanned data that we’ve collected. And then these gaps are filled in with the second layer of reiteration of information that we collect from site.
For this I need to skip a minute for the video.
I hope this works.
So for example, this is a software we are developing ourselves right now where we are merging the two without having mesh in between to interact. And the part that makes it really efficient is the high resolution that it allows you to take so you’re able to take, this is at least 30 million points that we have capture and more. So you’re able to get the same resolution throughout between the two, even if you zoom in and zoom out which is a little bit different than gaming technology, which is visible today in the field.
So what do we do then practically on site? And this may be helpful for people that are working directly with conservation authorities. We use five different techniques on site. And this has been steady since the first time we started doing this work. This is something extremely critical for our work so that we do not forget what is the overall intent of our field.
The first thing we do is we go and go through digital … Sorry, historic information that’s available about the site. It includes anything and everything we can find. Photographs, books, maps, we go into attics, we look into anything possible that can allow us to collect information about that site and it takes maximum time because it’s part of pre site visit information collection. This is done along with students so that when they come to the site they’re actually pretty aware of what they’re going to see, the kind of questions that past scholars have already posed, and what kind of things they should be looking for. So they’re pretty much well versed with the site before we get there.
Then when we get to the site, we do field notes like everybody in this field does. But it’s a crucial part of the work because once you do 3D scanning, you start realizing, scanning is great. It goes fast, it collects information for you, but it doesn’t know what material it is. It doesn’t have all the kind of nitty gritty information about oh, this happened here. Or look at this part. Or all the photographs that you would take about some specific kind of information you found on the site. The scanner is not able to collect that for you.
Then we also do photogrammetry. You’ll say, “Well, the scanner does all of that, why would you want photogrammetry to exist?” Well photogrammetry at a very high resolution allows us to take photographs that are very good in scale and then you are able to relate back to the scanned data and capture some more detail information for it.
And then we use gigapan and 3D scanning. We divide students on teams to be able to get back data collected. And then we come back. The on-site process is, we typically have very short time. So it’s 10 to 15 days only that we can spend on a site at a time because of the expense, because of you know, the conditions, especially when we are working on a site like Taj Mahal where there are millions of people. 25,000 people come there every day. They take your scanner it’s like, they will take a tripod and they will play with it. They will stand in front of your scanner and they’ll be like, “What’s happening?” And we’ll be like, “Nope, you don’t want to do that.” And you know, other things.
So the forum, people would be like, “Oh, this looks interesting.” And they would turn around our targets. And we’d be like, “Oh no, all our data. Hold, stop.” So we had a lot of fun experiences in the years that we collected.
So 15 days is the max that you spend on site directly, and then you spend a lot of time post processing, which is the longest. And this is the time we work on … We started the Taj project since 2008. We’ve been now working at least nine years and it’s still a work in progress so it’s a lot of information that we collect and give to the scholars that are in the field.
Now what I want to say is very critical here. While all of this is happening just with our team, it’s very important to be able to integrate this with the people that are actually working on site and doing conservation. One of the key problems I have seen over the years is that people that are working in natural conservation are extremely frustrated with the 3D technology because they are not the technology specialists. They want the advantages of the technology, but at the same time, they do not get the full benefit out of it.
The data will be put together, somebody will come. A team will come, do the scanning, take that data, and then give some kind of an output out of it and leave. They’ll give them the scan data but they don’t know what to do with it. And it becomes extremely difficult to work for them with that actual piece of information that’s been collected. So what we decided was to create a model, a way, a method by which you could find ways in which you could integrate the different needs of different people that are working on site.
So depending on the project, we consult and work with different departments. We work with the structural analysis department for the understanding of the structure of the building. We work with the department of chemical, you know for the chemistry department, the physics department, and other fields. Sometimes with anthropologists, archaeologists. Different people who will be able to give us outputs based on what kind of research question is being asked, what kind of conservation question is being asked. And this is all integrated into our 3D data as a final output that is given to them.
So let me start with the first project, the project that Kirk knows pretty well form the last time I showed. As I said before, it is in collaboration with the Soprintendenza there. Anybody who has worked in Rome before? Yes, if you’ve worked in Rome you’d know how it is to get along with the Soprintendenza. They hate foreign people coming to their country and getting some information out of them and leaving and publishing it. The reason for that is, and I was looking at that too, why is that happening? They themselves know a lot about that field. They know a lot about the forum. They know it in and out. They work there daily. And then, if we are working that long there, they feel frustrated that somebody comes and just takes a part of it and goes and leaves and creates an understanding out of it. So they want to be an integrated part of the learning curve and also the output that happens. So everything that we are publishing, creating, presenting, is a part of the partnership that we have with them.
This is 2010. In seven years now, already this is a bit out of date. And the data we collected was, we had to refine and understand how we are placing our scanner. If you know the forum, the forum is a place that’s very scattered and it’s got a couple of fragment monuments everywhere. The terrain is as important as are the monuments themselves. And we wanted to capture every stone that was existing form the imperial times. So we, to collect that kind of information, we had to make a plan, a map, of how we would be setting up our scanner around the site. So this is all pre site work where we identify and locate ourselves before we even get there. And we say what is the best location and positioning of the data to be able to collect maximum information. We’ve been there now at least times and now we have information that’s above ground and below terrain. Maxima Koloka is this kind of little drain that exists underneath the forum. And that was scanned with the scanner that’s a mobile on … It was constructed with a robotic camera to be able to capture that information.
So the data that is collected now is able to give you gauged understanding of the different layers that existed. Sorry for pixelation, is this visible? Is it clear now?
While we are covering scanned data that’s for a very high range, we are also looking at one millimeter accuracy information about the individual temples that are existing on site. So we were able to capture, here’s the temple of Saturn. And be able to see for example, every little detail that exists on its surface, including when the clamps that were having the metal lettering on top of the temple were taken out and so on, so forth. And some more glimpses into the information.
Along with the scan data, we also collected gigapans, which is high resolution digital photography merged into one. Gigapan is just one technique, there are multiple now in the field that do the same. It’s the same technique that you use for your panoramic picture when you take your phone out and you do a panorama, right? There are multiple pictures that are applied and connected together and gives you a panorama, same technique. Now imagine 500 of such images that are put together at a very zoomed in scale and gives you an image that is at least 17 gigabytes in size. When that happens you are able to look at a picture that’s full and at the same time you can look at the details inscribed in it that allow you to see small holes, cracks, damages, anything that you need to study and understand about that site.
So while we are using scanning, photography’s also coming very high speed at very good resolution. What’s that doing is eliminating some needs of the scanner at some point. And sometimes we use scanning or photography depending on what we exactly need as an output. For example, at the Arch of Septemeservus, which is right here, we were able to see graffiti that was put in from 15th century that was up there. You know, there was soil up to here before the whole area was excavated, so people were able to write here on the surfaces with graffiti markings. And with the photography we were able to actually examine all the graffiti marks that were put onto that surface. Just one of the outputs that came out.
Or for example, these gigapan images that allow us a full scope of the forum. And you can zoom in on any of these temples to a high detail. And they are available at around 130 gigabytes each. I’m mentioning these sizes because, later at the end of the presentation, I’m going to tell you one of the problems that this creates.
Thus finally we were able to use, put this to good use. We took the scanned data and were able to create plans, innovations, sections, information that could be used by the actual conservation team. So when we started mapping, creating the plan of the forum, we found out, for example, from conservation purposes, that only 70 years, the last 70 years … This is the basilica of Julia. The marble that was existing on the ground of Basilica Julia, which was very beautifully documented by the art students, is now into stone forms this big because of the kind of grass that is growing on that field now. And it’s percolated throughout and has completely destroyed the marble floor. So in a way, the scanning information allowed them to map and understand what has been conserved well and what is in a state of extreme disrepair.
Along with the scanned data we also created watercolor hand drafted drawings. Now why are these crucial? These are crucial because somebody who does not want to … It’s not somebody who doesn’t want to, but it’s available for people to be using as and when they need. This is something that is given to the government and it goes into the archives and whenever they need to, they can now compare this drawing with Jacomo Boni’s drawing a hundred years ago where they were able to see, okay this is what exists then. This is what exists now. And they are actually able to do their own new research in the field because of this informational available.
We also created hand drafted drawings that are part of the data that we collected. So we took the advancement of one millimeter accuracy information and then put that into something that is very traditionally available and understood by people from all fields. And these were then given as a part of resource to the Soprintendenza. So scanned data that we collected on site, line drawings produced from it, watercolors produced from it. The watercolor is a crucial part of line drawing just to understand gauge, distance, depth, and material.
Any questions till now?
Speaker 2: Did you elevate your scanner and your gigapan when you are doing it, off the ground?
Speaker 1: Yes, so we had it in multiple locations. We located it on different terrain levels at that time. We did not have the drone technology as yet. We just used balloon photography, at that time we did not use anything where we were able to raise it higher than the typical tripod level that you would have.
But we did locate it at different terrain levels so that we could get and capture the ground below.
What came out of this also was a good understanding of how these temples were located in relation to each other. It gave new information, this is getting published in a book of the forum, for empale, which allows for a better understanding of the core relation between the different temples that exist on site. For example, Temple of Saturn, which is right in the center, is the shortest temple but is located on the highest terrain in the forum. Whereas temple of Castor and Pollux here, the tallest temple, but located on the lowermost ground level. And giving a sense of the understanding during the hierarchy during this time for the temple building existed in the forum.
I’m going very quickly because I have a lot of things to show so please stop me if you guys have any other questions.
This is the second project, Taj Mahal, in Agra, India. The site is huge. We are working like crazy. If people want to help, please come.
Just to give you a sense of scale of Taj Mahal, this is the human being. You see that guy there, this person. So colossal, it’s just colossal in scale. The project is not just the Taj Mahal, the project is 46 tombs and gardens around the Taj Mahal. So the Taj Mahal you saw there, it’s this. This little tomb right here, do you see that? And this map is inscribed in here. And this is the holy Yamuna front and there are 46 tombs and gardens. I’m gonna retire by the end of this project, even with the scanning technology. But we’ve been able to, you know, go through many of these, if you can see, we started documenting these pretty seriously and what is happening is it is allowing for the conservation team to establish where exactly they will conserve to make this again, into a reverse site city as it used to exist before.
Again, the sense of scale, that’s a human being before. That’s the Yamuna River and the tomb by itself and then the larger complex beyond it. It’s marble clad, it has a core masonry, brick masonry structure, and in two thousand … What was this, 2014 because of the earthquake that took place, there was a need to understand and assess the structural damage on it. And so we were asked to come in, scan the monument, and understand if it was structurally stable or if there were any concerns noted to it.
So we did the same thing like we did at the forum, first before we went there we collected all our information. We created a map of how we wanted to document the site, and because we knew what was the goal of the actual conservation team there, we were able to understand okay how are we gonna go and map out the structural stability of this building? What we were doing is we were not doing the typical engineering part. The engineering part is done with the engineering team so I’m not gonna present that part. But I’ll just show from the scanning technology what are the results that we were able to put together.
So we went around scanning. There were some monkeys here that were also trying to play around with the equipment that we had there. It was fun. It seems so easy, right? This is a shot purposefully taken when there was nobody around us. But normally we also find locations that are a little bit away from people which includes, and this is where the scary part is, we went to the minaret on top here. The minarets do not having any railings and there are around a thousand bats in those minarets becau;se since 1945 it hasn’t been opened up so it’s very scary. And you’re holding a scanner, holding your helmet, holding your lights, you’re going into that minaret. It is not an easy task. Getting up there, only two people can stand. Actually one person still needs to be below and then the scanner needs to be there so then positioning yourself. And then you have to factor wind. And if there are errors because of wind to the scan data that you collect.
So finally with everything, we were able to get the information put together. The other problem was that the scale is so large that the scanners’ range doesn’t reach that far. So you have to have multiple intermediate scans that are allowing you to collect that information comprehensively and cohesively together.
The results produced allowed us to get some really good 3D models. You will see that the 3D model actually doesn’t look that good here. And that’s because the computers, the screens, the data was so much that they were not able to even pull up the full data in its full range to be expressed onto the screen.
Here are some results. You will see that there are gaps in this data and that’s purposeful. You will see that this is between one leg of the scanning to another. So then we judged these and then we cover up the next level.
What we did of course, in the end, this is work in progress project, but what we have done recently is we have given them a structural assessment report. And this goes back to you know, how are we integrating this with not just 3D scannings. We’re taking the 3D scan information and we have now put it into a report that allows the people who are working with us to get the knowledge we have gained from the site which includes understanding of the riverfront, what exactly we found out from the damage mapping, and the conservation issues we see on site.
So it included assessing and identifying what different types of damage exist on the site itself and how conservation efforts could be done to help that. Now this may look like little ants on that screen. I don’t know if the resolution is okay, can you guys see this one? If you see, please don’t photograph this. This is a highly controversial issue so that is why we do not want people to photograph this information. If you see the cracks on the surface, they seem pretty small but they are important and significant, including the one on this minaret here which is close to the riverfront. And that is an issue that is highly controversial today in the field.
And so the government wanted to understand and relate to is this a problem five years, no, not five. I think a few years ago in Smur Sion, was it Smur, Sion? You know Time magazine put up a cover, is Taj gonna collapse? It created havoc in India. And so since then, touch reports are kept highly confidential and it creates a problem of information being let out.
So if you ever, that’s one of the reasons we do not visibly show this in the crowds. You have to work with the government to understand how much of this information can be revealed and how much cannot. So we look through everything on the surface and we started seeing the ways in which conservation has been happening in the last years. Along with the scanning we were able to accurately identify where each of these areas were and tell them what kind of conservation effort needs to be created.
So by giving them, the first time, an overall picture of the whole Taj Mahal, which is scale-wise very difficult, this is the first black film on the surface to Taj and it already has a lot of cracks. Now imagine that on a large scale and you need that large scale understanding to understand how is the monument operating structurally? Is it okay? Is it stable? What kind of issues has it had? And it was a mammoth task to put this report together.
And literally we have every crack documented. Along with that, we also documented the etchings that we found on site. Remember when we said you do field notes, you’re taking photographs on the site and you’re collecting information and you see something that may not be captured by the scanner, how do you put that together how do you make that into a physical available information?
This is, for example, artist drawings that are made on stones on Taj which indicate which artist was, or which family was responsible. We say it as artist today and basically what was happening was families would put up their markings so at the end of the day they would get wages based on how many stones they had carved. And so each family had a little marking indicting what they had done for the day. And we did not want to lose these because right in front of us, while conservation was happening, these were being replaced by new stones. So very quickly in our report we had to identify them and this looks like insects but basically it’s the floor plan of Taj. And it has a little key there that indicates all the kind of signs that are part of the pavement pattern of Taj. If nothing is changed, at least we want them to be conserved, to be maintained if possible.
Last two minutes, Belvedere, we just started with this project. It’s 2016. When we started, that’s the Belvedere, it’s a huge project but not as big as Taj. And this was done by Bramante as an architect and so we’re fascinated to see what Bramante did when he built this work. When we started doing the scanning part, we realized an amazing fact came in front of us and I wanted to share this with you. It just came up two weeks ago.
This is the courtyard so right, this whole area is what we are scanning. And it’s divided now into two parts, this and this. And we’ve just finished scanning this area. We knew that, you know, there was a change in depth that existed here. We also knew that it was used in a theatrical purpose. But what we found out, which was pretty amazing, is this. Do you see these lines? The only line that’s parallel to, let’s say, ground zero, is the center line. And I know it looks perspectively ugh, what’s happening? But this line goes down so the ground actually goes down as you move forward. And then this comes up and so there is this kind of perspectival change in grade that happens at the courtyard. And this is still half of the courtyard so there’s some more that’s gonna come at the later stage. And you start seeing how each and every bay in this whole complex has been designed differently to be able to suit that grade change. And so we are standing in the original place to get a sense of the whole thing coming together.
Okay, so let me sum up. Like I said before, we do use at least five different techniques on site. We do off site evaluation of that information and then we create different outputs including different line drawings, watercolors, 3D model, 3D model with mesh, 3D model with image as an output. And these are then taken for various purposes to create refined understanding of the site. What we do see as a problem in the field right now is the data collection and storage. I know this is something a lot of times discussed, debated, people talk about it. But we still don’t have a very good solution for it. We’ve worked with Amazon, we’ve worked with Google, we’ve worked with Microsoft, no, none of them still. Google actually … No.
Then there is the data library management. The understanding of how we can manage the data that was collected in 2008, let’s say, to what we have collected in 2017. How do we map them, how do we overlap them, how do we make them relevant and visible and usable? Software continually a big problem. What was used in 2005 does not even get used anymore in 2017. Talk about 1999, it’s just completely different ways of working. And so continuity is a big problem in the software industry. Each time there is a new software everybody comes up with new things but they’re not talking about advancement through continuating.
Then large data management and hardware. Our computers cannot cope with eh amount of data we have collected. It’s very difficult. The computer just stalls. Even if we are using super computers, it’s that bad. Visual resolutions, when you have such a lot of data, it still doesn’t get converted into good, high impact resolution 3D models which, how do you cope with that?
And then finally, 3D interaction and practical use which is basically you’re taking all of this, how do you make it into an environment that’s not just 3D visually good but 3D usable as a space. So you can actually go in that 3D model, work with it, use it, and that would be the most amazing part where a conservator can go into the 3D model of a site, go in and understand whatever information they need to collect, grab it out of that, use it for their purpose and then be able to come back.
I end with Alberti, 15th century, where he says, “Never go away from what are the customs practiced by the best architects of the field which is not only drawing and sketching, but also models.” At this time he’s talking about models of wood, today we talk about completely different models and materials. These enable us to examine the work as a whole, as a full spectrum, not just one part of it. Not just specializing in it, but seeing it as a part of much bigger needs to be able to continue any further and to take away from any troubles that would come out of the work we have already done.
Thank you very much.
Do we have time for questions?
If any questions, I’m more than happy.
Speaker 4: How much progress have you made on being able to eliminate the mesh step? You said you’re experimenting with that. Is that a software code or are you building hardware interfaces? How does that work?
Speaker 1: Yes, so we’re working in software coding for this. We have our own software. Again, a problem, right? We are working with Cyclone, actually, which is the Lyca based software for 3D work. They do have their own bugs and problems so we’re working with them to create a way in which you can take the Cyclone data, you can convert it into pts, and then directly bring in photographs of the site without a mesh format necessary to structurally map that over onto the monument itself.
So it is successfully been done onto large monuments like I showed the arch of Titus on Septimus Servus. We are still working on the Taj. It’s a much bigger model, but we are finding issues still with ground terrain data because of the way in which mapping happens. Triangulation is one of the key ways in which mapping of the surface happens. And so we’re trying to eliminate that as a problem. The way we are working with the technique is the software actually wraps around the photograph onto the surface as if it’s a box and tries to put it together without the mesh so the photograph becomes the mesh in a way.
Speaker 5: You’re describing that last building, the foundation I think going around the distinct ground contour?
Speaker 1: Mm-hmm (affirmative)-
Speaker 5: Just makes me wonder, the foundations of these huge structures …
Speaker 1: Mm-hmm (affirmative)-
Speaker 5: Do you have findings about the foundations in terms of your reporting on structural integrity? Like how did they do foundations on the sites?
Speaker 1: Yes. What I’ve shown is maybe about one percent or a glimpse of what we actually understand about the site. Depending on what we use for questions, we pose particular questions regarding it. So if we have foundation questions, we will go and address, not just with the scan data, but with other field notes and so on, so forth, onto that site particularly for that. And then we are able to get new information around it.
It’s incredible how much information comes out. Sometimes it’s difficult to even just … You always filter it down somewhere. That’s one of the critical problems that exists. So when you’re working for example, when we were working with the Bramante site, with the great change, we found out that there were at least four or five new layers added on top of that lower surface from where it was originally. So originally they could have been steps that existed there. And so we’re still finding that out. It’s information that’s still work in progress. So you start getting all that work.
One of the other things we were looking at is ground penetration radar to see if we can take scan data to be integrated with ground penetration data because that’s another crucial technique that exists but it doesn’t have direct integration right now in the field. But we have a goal to be able to do that.
The scanner just works with what’s visible whereas ground penetration radar works with much more different ways for getting better sense of the site.
Speaker 6: That was kind of one of my questions, coming from the cultural landscape world of course, I see the buildings, but when I look at the model and river scape and everything, how do you negotiate the structure? So I’m just kind of wondering how you do, don’t integrate that?
Speaker 1: So the lidar flight technique and the drone technology, both of these are very helpful for these things. So for the Taj site, that’s one of the things we are using because of the scale and of course the terrain. Lidar works very well with flight techniques though the resolution is not as good.
Speaker 6: Have you been doing video and grabbing still images or do you do still and capture and piece them?
Speaker 1: Okay so when you use video you can get some amount of information but the resolution is not the same. Our goal has always been to go up to the accuracy of one millimeter and that means a lot of data for us. So again, depending on what exactly the goal of that is, so let’s say you’re looking at the riverfront and we are looking at the riverbed, it doesn’t need one millimeter accuracy. And in that case you can do you know, still imagery, video generation, drone scanning, or lidar scanning. All of those work pretty well for those techniques.
In the end we integrate both of these to the high resolution ones so we are able to get data captured at both levels.
Speaker 7: There’s a lot of photogrammatic software out there automatically being integrated to image, higher resolution …
Speaker 1: Mm-hmm (affirmative)-
Speaker 7: Radar scanning. So have you ever tried to enter the building?
Speaker 1: Yes we have, and we’ve looked at that. That was one of the things we were doing with the mesh. Before we started even the meshing search, why exactly do we need to do this? Because if there’s’ already an understanding existing, are we replacing, replicating? The study showed us that the accuracy’s not as high as you’d think because it’s relative. Because if you’re taking a photo …
Speaker 7: It’s [inaudible 00:51:15]
Speaker 1: Yes, yes. Because it always, it starts with the basic principle of how does the meshing happen or how does the integration happen? So let’s say you take a photograph here and then you take three others that are there. They’re always gonna find the best fit. And that best fit leads to you know, it’s tilting or being able to adjust the accuracy to be able to do that. And that changes the way in which the information that is going to come back to you.
So for example, at the Bramante site where we saw this angle change, if you would have done that, we would not be one hundred percent sure where the zero ground line is because of that change. So you then need to do that and then integrate it back into 3D scanning and then you are able to get the full scanning. So it’s doable, not saying it’s not, but it always depends on what’s the best efficient method at that time for your work.
Speaker 8: For the archival aspects of it, to what extent are you or are you using e57 compliance?
Speaker 1: That’s a very good question. We are working with … Because we are working with government organizations that have their own specific rules also of how they want the data to be put together, we first rely on what they want. And then we have our own little system to build with it and we do not comply actually. None of those are … We’re working … Let me just say it like this. We’re finding our own methods right now to build it and work with it instead of … WE try to work with very, we talked with Getty for example, for understanding how they’re formatting works and how … And we still saw problems associated to that and so we’re not.
Speaker 8: So the E57 is the …
Speaker 1: I’m not aware of the E57, I’m sorry.
Speaker 8: E57 is a neutral file format that now is industry standard for 3D and so each scan from no matter what software you’re using, we’re recommending that you export each scan as an E57 compliant for an archival purpose.
Speaker 1: Mm-hmm (affirmative)-
Speaker 8: And that way, no matter what the software is, whenever it is, it’s a neutral file format.
Speaker 1: Mm-hmm (affirmative)-
Speaker 8: And maintains the importance of that is that it actually remains the reflectivity of file fields and a lot of them, like if you save it as XYX or Aske, it doesn’t maintain the reflectivity …
Speaker 1: Right.
Speaker 8: That E57 does and so you might want to look at that. I’m just thinking for archival
Speaker 1: This is great. I would love to look into that to understand.
Yes, yes, yes, I think that’s very good news for me because … And that would be a great way for the conservation teams, too, to be able to work with it.
Speaker 8: Right.
Speaker 1: So I wasn’t aware. Three years being in Rome doesn’t help in that situation.
Speaker 9: Yeah, I had a question you dust mentioned the conservation team and you said early in your talk, I’ve certainly experienced the same thing where heritage conservators who are actually in the field had expressed their frustration with using his data. And you said you had the one slide to show the research and practical conservation you couldn’t find it, but it’s a much more integrated approach. And so I wondered if you could speak a little bit more to that and are the conservators on the ground actually using a digital lab and not just products that are generated on their own?
Speaker 1: Mm-hmm (affirmative)- they primarily, everybody uses the products that have been brought out but the digital team members work with us. We sit with them literally. And because they are not so aware with how the technology works, they’ll always, it’s interesting, sit at one of those rooms in the Vatican and tell them, “Here, take a look at the 3D data,” and they’ll be very happy and excited about it. But tell, “Let’s have you operate it, work with it,” we have someone from their team actually train with us so that they are better able to understand how to work with those.
We also do is we sit with the chief person who is the conservation responsible person to say, “What exactly do you need? Can you give me a sense of, okay, what …” and we make them walk through and then we actually will say, “This is what you need or this is what you need,” and we will produce out what’s based on that for them too.
But again, this happens till the time we are on site. We wanted to continue and be useful for other people.
I don’t want to say too many stories but at the Vatican for example, and at the Taj, there were people who came and did 3D scanning before. We are not the first people to scan it. And at Taj, it’s very sad, but they came and they put up these big stickers on top of the surface to be able to get these targets. Those stickers, when they were removed, had edices on them and created marks on that surface which was a disaster. And when we went there, they were like, “You are scanning?” So they already got into a negative place with us. They were like, “Oh no, we don’t want that. You guys are gonna, do you see those?” And I said, “We don’t need those. We don’t use targets anymore. We don’t need those at all now.” And so they were like, and they spend a lot of money to get that company to come and do that work with zero output. And that’s the sad part of it. That there are enough people out there that they’re doing their job but they’re doing it incompletely and that’s causing bad reputation for 3D scanning. Same with Vatican.
Speaker 10: You said you do not use targets then?
Speaker 1: No, we don’t.
Speaker 10: Are they obsolete?
Speaker 1: Not necessarily. It depends again on the project, but because of the scale of our projects and because of the kind of conditions we have, we cannot work with targets. They actually create more errors in our work than help.
Speaker 10: Putting it back together?
Speaker 1: Putting it back together, yes.
Speaker 11: Are you using an algorithm to do the mappings or drawn maps or is somebody visually working?
Speaker 1: We do both. We do the algorithm part and we do visual understanding because I think both have value to them. One tells us precision, the other tells us information regarding the type of crack we are working with.
It’s two different teams.
Speaker 12: Do you know if your data that the conservators have seen, do you know if it’s changed their approach to anything or how they fix it no?
Speaker 1: Absolutely. More than just the data, it’s actually the final reports we produce which go to them and they understand. First of all, they realize we’re working with them, we’re not working against them. And then secondly, they start seeing the benefits of it and every time they’re doing something they’ll tell you, “Can you send us this, can you work with us on this?” So it works out in a very beneficial manner. And so finally when we write the report, we write it in a very clear manner without being you know, nice to them, but just telling them, “Okay, this is what the problem is.” And they accept it. They are pretty accepting. Especially at Taj when we told them, “There are some efforts that are taking place that are causing problems including the use of epoxy for inlay work.” That’s a new thing they had started doing ’cause somebody told them it’s cool to use it, so there they go. And then we started seeing cracks with those.
And so when you show them the results and you show them what has happened, they are much more aware and accepting of it.
Speaker 12: To document at the larger sites like Taj Mahal, to teach the drone to that because of it?
Speaker 1: The drone is gonna be, could you repeat the last part please?
Speaker 12: Yeah, drone cannot be a way of to draw to document the larger site.
Speaker 1: So drone and the car scanner, let me say a few words about it. Or the backpack. All these are techniques where you’re having … Or lidar … Is moveable technologies, techniques by which you can grasp information fast in a mobile manner. What it’ll do is it’ll collect a lot of data for us. These are just points. What you do with it is a complete different question. So while a lot of data will get collected, what exactly are we gonna do with it? I can share with you a world that is gonna change and transform internet. We work on 2D platforms, we will have 3D platforms on the internet soon with the amount of data we are collecting. But it’s just a question of how quickly the technology moves with not our field, with not scanning, but with a complete different team of people like Google and Microsoft who are taking new ways in which we experience the world.
I think that’s it. Thank you.
In the last 15 years huge advancement in the use and availability of 3D documentation technology has taken place. From a very bulky scanner to a variety of ways in which the 3D data being captured have now been explored and continue to evolve the technology in the field further. While the technology has evolved on the hardware and software advancement, the method by which it becomes more available to various parties involved in conservation of the site is still evolving at a snail’s pace. This paper concentrates on how 3D technology which was once the field of a specialized few, is now evolving into a very user friendly technique of working on the site. Also with it through the projects of the DHARMA (Digital Documentation of Architectural Research and Material Analysis) team have led to new means and methods by which the gap between output of 3D results to utilization for end user are being reduced with each project.
With projects like the Roman Forum, Rome, Italy, the Taj Mahal, Agra, India and the Belvedere at the Vatican, the paper will present the interdisciplinary approaches taken to merge the results of 3D documentation to create comprehensive end products that are directly useable by the end user for protection, conservation or restoration of the site.
With interdisciplinary teams from architecture, computer science, archeology etc, the output is custom tailored for the specific users working on the project. The scale of the project goes from microscopic to macroscopic allowing for a variety of data to be collected and comprehensively combined.
The paper ends with issues that are still open in the field and are being worked on as a part of the evolution of the overall approach in 3D documentation of heritage sites.
Krupali Krusche is a preservation and architecture faculty member at the University of Notre Dame. She started the Preservation concentration at the school in 2007. As the head of the DHARMA (Digital Historic Architectural Research and Material Analysis) research team, specializing in 3D documentation of World Heritage Sites, she has studied and documented key monuments including the tombs and gardens along the Yamuna River in India, the Roman Forum in Italy and recently the Belvedere complex at Vatican. She has numerous articles published and under publication based on these subjects. Two international exhibitions are planned for these projects, one of which was planned to be the first 3D exhibit on historic preservation in the Roman Forum from April to Oct 2014. In Nov 2012, her first co-authored book, “Rediscovering the Hindu Temple, Sacred Architecture and Urbanism of India,” was released which is based on documentation and proportional studies of Hindu Temple in India. She also has contributed chapters to books like, “The Venice Charter Revisited,” and “Durability in Construction,” etc.