This presentation is part of the 2017 3D Digital Documentation Summit.
Combining hands-on inspection, nondestructive evaluation, and drone-based data acquisition in the investigation of the Soldiers’ and Sailors’ Memorial Monument
Doug: Hello, good morning. Thank you for having us. So I’m Doug, this is Kristen. We’re gonna speak to you about a project that involved a lot of technology and allowed us to do a bunch of cool things. There are a lot of challenges to surveying this project. We got to use a lot of digital technology. We spent a lot of time on 3-D workflow, it was important for the project. And then coming out of this project we started thinking about ways to do things a little differently in the future.
So I’m gonna give a little background and then pass it on to Kristen from Vertical Access who’s a big partner for us. Background to the project is that the Soldiers and Sailors Memorial, which is located in Manhattan in New York City, is owned by the Department of Parks and was in need of a full condition assessment. They turned to New York City and Office of Management Budget and [inaudible 00:01:09] Architect had a prime contract with them and we were on their team. They asked us to put together a team that could do a full condition assessment and also restoration treatment study for future to try to understand the magnitude of what would be needed to fully restore the monument and the memorial, actually.
We led a team of Vertical Access, GBG, and other consultants to do the assessment. Access to the site was actually very difficult, we’ll talk about that a little bit but first a little background on the monument. It’s actually a memorial, Soldiers and Sailors, built turn of the century. Architects were Charles and Arthur Stoughton. It’s a classic monument, Corinthians temple style. It has a high base, it has a colonnade, it has a low conical roof. It’s a central part of Memorial Day celebrations in New York City, ever since. It actually houses quite a prominent location. It’s in Riverside Park on 89th Street. It actually is built on top of a natural rock outcropping in the park so the 89th Street is actually quite high. So it’s very prominent. It’s a New York City landmark. It’s surprisingly how much it’s used even though if you walked by on a regular day you may not see a lot of people. It’s an important monument to New York City. Despite that, they did not invest in it too much over the years.
New York City as you many know has grown and Manhattan, the coastline has extended with fill. But fortunately, even though tree growth has, you know the Parks Department has allowed trees to grow on the other side of the park. It’s still quite prominent from any location.
So to give you a sense to what the monument itself is, and the talk today really is about the monument, is the memorial includes a series of plazas and staircases that surround it. But the monument was the critical thing because of its height and the fact that the public can go around it and ideally inside of it. You can see it’s actually, it’s a series of mastery walls, with exterior stone cladding, the base is granite, there’s a lot of marble. There’s marble in the interior chamber as well so basically you walk up a series of steps and you walk inside through a bronze door and you look up, and what you’ll see is, and you’ll get a better sense of this photographically from Kristen’s talk. You’ll see a Guastavino vault that’s an oculus as an opening inside, has mosaic on the soffit. And then as you rise up, you can’t really see what’s on the side’s above, but there’s another Guastavino vault that’s mosaic as well. So there’s some openings that let light in, so it’s actually quite beautiful when you come in.
We’re very fortunate that Parks Department did have drawings, they had drawings from the original 1900 construction, not a complete set. We actually did not have good drawings of the monument itself, but we had the original drawings for the plaza esplanade. There was a major restoration in 1960 and we did have a drawing set associated with that. We did a lot of archival research, a lot of photographs, a lot of evidence in the newspapers. The thing we learned quite quickly was the minute it was opened and put in use, it had problems. It was leaking, they could describe the mastery as sweating. We had a mortar joint falling out. So for many years it’s been a problem. In fact, so much so that in the late 1950’s had to temporarily close it, as you see there. The other important fact is that many of the symptoms we saw in the photographs and documents and newspapers, we saw when we did our survey. So the same thing was happening all over again.
So New York City, the Major Restoration campaign in the early ’60s, they redid the entire series of plazas. The conical roof, the finial, has marble slabs, they actually put lead sheets on top of that. They removed some interior drainage and did a complete mastery restoration. And that slide on the upper-right you see a bit of what the site looks like. Basically you enter from 89th Street right here, and these upper plazas are quite narrow. And there are staircases that go down the slope of Riverside Park.
So with that. . . whoops wrong direction.
I’m gonna turn it over to Kristen to talk about how we actually surveyed this.
Yeah so Doug just gave a really nice succinct description of some of the challenges with this site and the biggest one was access so you can see that on these plazas, there’s a very narrow area that goes around the monument. And what that meant was that with a lift, you really could have only gotten access to about 25% of the exterior. There are no ladders on the exterior, nothing on the interior, no hatch to get out on the roof, so really it was figuring out how to get on to the monument from the outside-in. So we did bring a lift on to the plaza and that allowed us to install some rigging at the roof in order to use rope access and so, that’s what you see here in this image. Having the anchors at the top of the roof and then hanging the ropes on there allowed people to climb the ropes up from the plaza level. Usually we like to, you know, go out on to a roof and then use gravity to our advantage instead of having to fight against it but it didn’t happen in this case.
So our anchors then consisted of encircling the finial at the top of the roof with padded anchor ropes and we did build a custom ramp to bring up that lift up on to the plaza. We were very careful not to overload it or damage the plaza in any way.
So this approach allowed Vertical Access and other rope access technicians to observe all of the areas of the exterior and get hands on to most of it. The areas once you get down passed where the columns are, we didn’t get hands-on to that interior wall but we got pretty good visual on the entire structure. And this voided impacts to the plaza, it voided having to install tie-backs for scaffolding and also was a minimal visual impact to the monument during the inspection, because as Doug said it’s a really prominent location, there’s a ton of people in the park all the time.
So this also allowed other team members from GBG Technics who are rope-access certified to get on to the structure to do some ground-penetrating radar as well as ultrasonic post-velocity testing. And then access was also a problem at the interior where again there were no interior ladders and we couldn’t get to that space that Doug described as above the lower dome with the oculus but below the upper Guastavino tile dome. So in order to avoid having to build scaffolding at the interior which would have been costly and better suited to full restoration work later on, we flew our drone through that five-foot diameter oculus with a very skilled pilot (laughs). And we were able to gather both video and infrared imagery of that interior space which you really can’t see from below because of the geometry of the oculus.
Let’s see. . .
So. . . sorry
So going back to the other challenge of the survey which was collecting and managing all of this data, it’s not a huge building but there was a lot going on and it was really important to gather comprehensive information about all of the existing conditions in order to start diagnosing the causes and what the potential solutions would be. So rope access really is just our means of getting where we need to be on the building, and then it’s the tools and the technologies that we can leverage on-rope that are really what helps us start to understand the structure. So this very thorough documentation of existing conditions was required here at Soldiers and Sailors and in order to do that we used direct digital documentation.
So this is an example of one of the photographs that we took where our documentation system automatically renames the photos with the numerical prefixes the X,Y coordinate where it’s located on the drawing, the material, the condition, the failed Dutchman, and the year. And that’s all customizable with whatever kind of information you need to have in that photo name. Our deliverable included over 1,800 digital photos keyed to over 1,000 annotations on the exterior elevation.
Sorry, I lost the mouse. There we go.
The system that we used to digitally document conditions on-site is called TPAS- the Tablet/PC Annotation System. Vertical Access developed this as an add-on that runs within AutoCAD, it’s a proprietary software that we have and basically each annotation is a database entry where as you’re surveying, you’re prompted to enter any number of data fields about the condition describing the amount, the severity, you could assign a repair, and that’s totally customizable in terms of what kind of data you need to gather about each individual condition. This vastly reduces the amount of time that we spent in the office preparing a report, eliminates transcription errors, it’s a way to make sure that everyone on-site who’s doing the inspection is sort of using the same nomenclature and that’s something that we work out with our clients before we get to the site, what are the data capture needs and how can we customize the system in order to achieve that?
So, you know, this means there are 1,000 annotations that we didn’t have to transcribe from paper into digital and almost 1,900 photos that we didn’t have to manually rename or come up with a system for naming those photos. Using the drawing in a digital format lets you really focus in on different conditions by isolating different layers. So right here we just see all of the crack conditions so we can start to identify some patterns and maybe some causes of the damage. And Kate said yesterday that they started using Revit when their clients started to demand it, so right now we’re still working in a two-dimensional AutoCAD environment but I think pretty shortly, maybe within a year, we’re gonna see some of our clients start asking for TPAS deliverables in three-dimensional format so we have a plan for how we’re going to make that happen, so stay tuned for that.
And then all of the data that’s embedded in these annotations, which were basically database entries, are exportable to AutoCAD and you can start to sort the data, you can assign repairs based on certain criteria and whatever you really need to do. The background drawings for this survey where prepared by LERA based on the existing drawings which were pretty good, but not complete, as Doug said. Then several non-destructive evaluation methods were also used to confirm the sub-surface conditions that were observed at the exterior and the interior so ground-penetrating radar that you say those two GBG folks doing, to help identify embedded metal, and they also did some ultrasonic post-velocity testing to assess the internal integrity of some of the mastery units.
GBG did infrared on the exterior from the bucket of the lift and again they were only able to do that on about 25% of the exterior because of the limitations of the plaza. This is where using a drone would have been really an ideal platform for getting infrared data and if you’ve done infrared you know that you wanna have as close to a 90 degree or straight-on angle of view as possible in order to get the best data, so standing on the ground and looking up at something tall, you’re not really gonna get usable infrared data. This monument is located in controlled airspace and it was prior, or just after we were on-site just after those part 107 rules went into effect so we weren’t able to request an airspace authorization in time to use the drone on the exterior, but the FAA does not regulate interior space.
So just as rope-access is how we get on to the building, how we physically get to where we need to go, the drone is really just a tri-pod and it’s the payload that matters. Our first drone, a DGI Phantom 2, was mounted with a Go-Pro camera on motorized gimbal and both of those were sold separately. The newer DGI models have an integral camera, as John mentioned, and shoot really nice video.
There it goes, right through the oculus, so now we have eyes on this interior space that I guess there were a few photographs from the 1960’s but we also really needed to see the space in order to see what the condition was of it now. You can see some of the cracks through the brick masonry and some of the materials that were left up there after the last restoration.
Then we also mounted a FLIR View Pro camera on the drone, which is a camera made by FLIR that’s meant to be carried by a drone, it’s very small. We used that to get infrared imagery of the interior. We scheduled it after pretty heavy rain in order to increase the chances that we would see some thermal differences in the masonry due to water infiltration and we did see cooler thermal responses at the joints, which was consistent with the failed mortar that was observed at the exterior and the interior.
One lesson learned here is that a lot of this technology, both the drones and even the program Pix4D that we’ll also talk about in a second, some of it is really developed for precision agriculture or larger site surveys so the infrared camera is factory set to focus at infinity, which works as long as you’re about 15 feet away from whatever you’re looking at. But inside this pretty narrow interior space we couldn’t always maintain that distance so we did end up with some infrared images that were a little too blurry to be very useful.
Something that we didn’t use here but that John gave a good idea of using, you’re using Pix4D also it looked like. So this is just another option for using photogrammetry where we take photos from the drone and then they’re stitched together into a pretty good model that we can use to create an ortho-image that can then be traced in AutoCAD. And for us we see this as a great potential for getting really good background drawings for our TPAS surveys so we were lucky on this project that LERA made very nice drawings showing joint-lines and showing a lot of detail. And that’s important because when we are using TPAS, if you draw a box or you draw in a crack, the software automatically calculates the area or the length so in order to get really good, accurate data out of the survey it’s important to have a really good, accurate drawing, especially if it’s a masonry structure or a terracotta cloud structure, something where there’s any chance that you might be looking at replacing entire units if you can visually see that where the joint-lines are then it’s much easier to make those decisions in terms of conservation.
I’ll give it back to Doug for some of the 3D workflows.
Doug: So one of the other things we were asked to do by Parks Department in New York was advise how the monument would perform under different load states. So we had to do a structural stress analysis. Being that it’s a masonry structure, there’s no metal, it’s all just compression and mortar joints. It’s stone and brick, essentially. We wanted to accurately model the transitions in the masonry thicknesses, the vaults, and so-forth. So we really wanted to, if you have any experience making 3D analysis models out of finer elements, its laborious. And for us we wanted to try to optimize our process and one thing LERA’s been doing for a while is trying to optimize 3D workflows. We deal a lot with our other types of work with complex geometries and so automation or coding and scripting processes is part of what we do.
So here we have good CAD drawings because we had pretty good original drawings, hand-drawn, beautiful hand-drawns. That was a great basis for TPAS and for other work that was done, but to create a 3D model we needed to be in 3D and actually, originally we explored using 3D AutoCAD, taking the 2D, making it into 3D, we found that a little cumbersome to then translate because ultimately what we’re doing is translating geometry for one platform to a structural analysis platform. So we decided to use the geometry information we had in 2D CAD, let’s generate a Rhino model If you don’t know, is a 3D geometry software platform that’s pretty easy to use, it’s used a lot in the architectural design field. We’re very familiar with it so we generated a rhino model of the monument as you see there. Wanted to go from Rhino into SAP which is Structural Analysis Finite Element Program. To do that, you can’t just copy it, we actually had to take each part of the geometry and translate it through coding. We use Grasshopper a lot. Grasshopper is a graphical algorithm editor, it’s a graphical platform, contains lots of subroutines or algorithms in it and then you combine them. And so you can write code in it as well.
So we have a bunch of these custom built already that we use a lot and we made more for this project, so we essentially took different parts of the Rhino geometry, which wasn’t necessarily complete, we informed it further. For instance, we knew what the base of the monument was like, but it was also all these voids in there so we kinda had to add that in. So taking each component and using programming, we were able to build finite elements in 3D that we felt were accurate enough to capture all the different stress flows. The monument is largely symmetrical but there’s the entrance. So there’s niches, there’s voids, at the end of the day we wanted to feel comfortable about stress concentrations and that there wouldn’t be areas that would have the stress that we were overlooking.
So all the vaults were created separately. There’s a base vault of brick, there’s three Guastavino vaults, no I’m sorry there’s two Guastavino vaults, one’s the oculus and there’s a vault on the very top as well. All those were created together to create a finite element model that we can then do various stress analyses as you see. And from that, we can then understand gravity loads, which as expected the monument behaves very well which correlates with what we observed. Most of the deterioration in the monument is actually due to environmental factors. We were also able to talk to Parks Department on how it might perform in an expected code earthquake. So we were able to find good results and look at stress concentrations throughout.
So we were able to generate plots like this. In terms of scale of effort that was about maybe a two-week effort to do. And sometimes that can take a whole lot more. Once we did all that, went through the process with Vertical Access and TPAS, we started thinking internally about what more could we do next time around? That’s part of our culture because we have a lot of programmers, we have people doing VR, we have people that are thinking about pushing things into the technology. To be clear, our use of technology is mainly on the software and programming side, we’re not doing the physical technology like scanning and the drone use. So we wanna work with partners. And one thing that was very interesting for us was that on the exterior we had TPAS and that was hands-on and digitally annotated. On the interior, we had a drone and it created all these visuals. Essentially we had video and we had still photography. And then what we did is look at those on the computer screen and then annotate in CAD our drawings.
So we were debating on how can make that process a little easier for us? So we’ve been playing around with coding that would essentially allow us to using differences in light and darkness on a video image or a still photographic image, identify those contrasts and try to understand which of those were cracks. We have people that are familiar with machine learning and so we started a process where you can first, with input, trace over a crack, a spall, whatever, erosion area, and then teach the code that that’s what that is. And it’s a process that works upon itself.We’re hopeful that maybe in the future we can get projects with sufficient scale and we can build this as a custom algorithm that we can do that.
So instead of someone being at the screen all the time, it can be someone at the screen watching the process, updating the process, and teaching it. Now it’s a bit of a tricky debate because there’s nothing that replaces hands-on. So what was done outside on TPAS in many ways is similar because you had digital annotation and you had the visual. So we think it’s appropriate for places where you really can’t get the hand-on as easy and yet you still want to optimize your timeframe. This is an example that we’ve done after the fact, that holds some promise for us.
Basically what we’re finding is that when you’re willing to do coding or programming, possibilities are endless. We’ve been talking about a lot of data, it’s really how you use that data and manipulate it from platform to platform and the only way to really do that is to do your own programming.
Now to take it a step further, and the reason why I show this up here is because a couple of years ago, people at LERA became very interested in Virtual Reality. A gaming background, whatnot, that became interesting using Oculus, HTC Vive and Samsung and so-forth, and there’s been a growing trend in the architectural design industry to use that to help visualize things. But we’ve been thinking about how can it help the design process. And so I challenged those people with how can it help with restoration work? So what we did on Soldiers and Sailors is we flew a drone and we took video, but we didn’t take 360 video, but you can do that now. So if you, and these are possibilities now, if you can do a drone, take the 360 video, take that, you have a 3D space and then use a Virtual Reality environment, you could potentially
. . . let’s see if I can run this now. . .
Work in a 3D environment, which we feel is intuitively a better environment to be in. Once you get used to the headsets and you have your systems working, it’s actually quite nice. And so instead of someone annotating on the screen, and this is while they’re using machine learning or some type of enhanced software or not, you can be in the space and get a better perception. That’s the one thing about video and photography is that it’s still a 2D perception on the screen, but when you’re in a VR space you get a sense of depth which is also important. So ultimately this is really not to supplant the hands-on but sort of get closer to that feel, so for the work in the interior we never got to go up there. The exterior we did and I think the difference between the two can be telling in some projects. Nothing is going to replace the hands-on but these are strategies for getting closer to that.
With that, that’s our presentation. Thank you.
Recent investigative work at the Soldiers’ and Sailors’ Memorial Monument leveraged a variety of digital documentation technologies, including direct-to-digital conditions documentation, nondestructive evaluation, and data acquisition by drone in order to allow the project team to fully understand existing conditions and failure mechanisms. Challenges included limited means of access for personnel, in-complete as-built drawings, and data acquisition and management for a structure in need of extensive evaluation to determine the extent of restoration. Additionally, the project team worked to limit the impacts of “downtime” for a public monument in a busy city park.
The Soldiers’ and Sailors’ Memorial Monument is located within Manhattan’s Riverside Park. It was dedicated on Memorial Day, 1902, as a monument to Union Army soldiers and sailors who served in the Civil War. Architects Charles and Arthur Stoughton won the design competition for the monument, with an entry based on the Choragic Monument of Lysicrates in Athens, Greece. Rising 96 feet above the surrounding plaza, the monument consists of a tall cylindrical base, a rusticated drum ringed by Corinthian columns, an ornamental entablature, and a low conical roof. The exterior is clad in white marble with a granite water table. The marble-clad interior space features a tiled dome with an oculus opening to a second dome, both constructed by the Guastavino Fireproof Construction Company.
The last major repairs to the monument were undertaken in 1962. The exterior masonry is currently in poor to fair condition, with widespread spalling and cracking of the masonry, mortar loss, failed Dutchman repairs, and failures of the sheet metal roof and flashings.
Working with the New York City Department of Parks and Recreation and architect Perkins Eastman, Leslie E. Robertson Associates (LERA) retained Vertical Access LLC (VA) and GB Geotechnics (GBG) to inspect the exterior of the monument. VA performed a hands-on and close-visual inspection using industrial rope access, documenting conditions using TPAS, the Tablet PC Annotation System, linking photographic documentation and quantitative data within a single AutoCAD drawing. VA assisted GBG with lift access in order to acquire infrared imagery of the monument exterior, and VA provided rigging and guiding for GBG rope access technicians performing ground penetrating radar and ultrasonic pulse velocity testing of the drum.
Vertical Access was also asked to perform a visual and infrared investigation of the monument interior using its Unmanned Aerial Vehicle (UAV, or drone). There is currently no means in place for personnel to access the interior space between the two domes. In order to avoid the erection of pipe scaffolding at the interior, which would be better suited for a full restoration program in the future, VA’s solution was to fly a lightweight quadcopter through the 5’-diameter oculus in the lower dome, providing video and infrared footage of the interior finishes. VA used a DJI Phantom 4 quadcopter mounted with a GoPro Hero 3+ (?) for video capture and a FLIR Vue Pro thermal camera designed for UAV applications. Conditions observed during the drone investigation were recorded in AutoCAD using TPAS. A live-feed video downlink allowed multiple project team members to observe the conditions in real time and to direct the drone pilot to areas of interest. The project team was also able to schedule the drone investigation immediately following several days of rain for the best opportunity to see thermal differences in the materials due to water infiltration.
The data gathered at the exterior and interior of the monument is being used by LERA for a full restoration condition assessment with scoping recommendations to determine a comprehensive cost estimate that facilitates New York City’s Department of Parks and Recreation fund raising for the eventual restoration of the Memorial. In addition, LERA utilized the data obtained to develop a 3D geometry model of the Monument using Rhinoceros to assist with quantity determination and structural analysis. The Rhino model geometry was manipulated with scripting within the Grasshopper platform to create an accurate finite element analytical model for further verification of the Monument’s performance under deteriorated conditions. The team is currently investigating the use of virtual reality to explore better communication of actual conditions and proposed remediation of improved access to the Memorial as well as improving future restoration investigative and descriptive tools with the use of 3D drone video capabilities.
Douglas P. González is an Associate Partner at LERA. With over 25 years of experience as a Structural Engineer, Doug has developed the design of new structures and renovations for a wide array of healthcare, government and cultural projects. In addition to his expertise in healthcare, cultural and civic facilities, as well as adaptive reuse and historic preservation, he serves as President of the Structural Engineers Association of New York (SEAoNY), having previously served as Secretary and Director. Doug holds a Master of Science from Cornell University and a Bachelor of Engineering from The Cooper Union in New York City.
Kristen Olson is an architectural historian with Vertical Access and a SPRAT-certified Level II industrial rope access lead technician. Kristen has worked on investigation and documentation projects including McKim Mead and White’s Manhattan Municipal Building, the Chapel at the United States Naval Academy and the historic campus of Harvard Medical School. She is actively involved in the continued development of the TPAS direct-to-digital documentation system and the production of drawings from video captured using unmanned aerial systems. Kristen has a Bachelor of Arts degree in Art from Colby College and a Master of Arts in Historic Preservation from Cornell University.