On-site with Spot: Robotic refinement of workflow procedures

Foster + Partners’ Applied R+D (Applied Research + Development) group has for many years been actively researching and using robots in various capacities. This interest led the practice to be the only architectural firm to participate in American engineering and robotics company Boston Dynamics’ Early Adopter Program for ‘Spot’; exploring how their semi-autonomous quadruped robot could be used in the architecture, engineering, construction, and operations (AECO) industries. Here, the team assess Spot’s potential to radically improve workflows, speed up processes and revolutionise Foster + Partners’ digital twin efforts.

Its extraordinary capabilities make Spot an invaluable resource as we attempt to rethink and reinvent our construction processes.

Foster + Partners’ Applied R+D team have been working alongside Boston Dynamics to experiment with Spot in dynamic environments such as construction sites; capturing changes regularly, and assessing the robot’s capacity to facilitate the comparison between ‘as-designed’ models against ‘as-built’ realities. We wanted to use the robot’s ability to perform consistent, semi-autonomous scans of construction sites to close the gap between the digital and physical states of a building which can often prove disparate.

During our investigations, Spot has been used on a range of sites. Starting with the Foster + Partners’ London campus, the robot dog was used to consistently monitor and capture on-site progress during recent renovations. The Applied R+D team then assessed the results of Spot’s expeditions to see how they could create faster and more adaptive design-to-construction cycles.

Constant and quick checks to monitor accuracy and timeframes, or how well we conform to set deadlines, can be instrumental in integrating design and construction, allowing for further flexibility. Additionally, we have been investigating how Spot’s scans could be of use not only during the design and construction process but also when the building is operational.

These repeated scans of construction sites, or “construction twins”, are four-dimensional digital representations showing how a construction site changes over time. The robot’s ability to repeatedly and effortlessly complete routine scans in a dynamic environment proved invaluable not only in terms of consistency but also in the large amount of high-quality data collected.

The robot’s ability to repeatedly and effortlessly complete routine scans in a dynamic environment proved invaluable not only in terms of consistency but also in the large amount of high-quality data collected.

Through this process, we gathered a sequence of scans that could help us track the project’s progress against project timeframes as well as facilitate regular comparisons against the Building Information Model (BIM). The scans can help ensure that rapid, and most importantly accurate, changes to the newly-designed system can be made to accommodate the differences captured by the scans – all in a matter of days. This could result in critical savings both in terms of time and money.

As previously mentioned, Spot’s abilities were also put into practice at Foster + Partners’ London campus, where it continued the work of building a construction twin by tracking the progress of renovations undertaken in one of our design studios. In addition to exploring the idea of four-dimensional scanning and its relation to spaces that change over time for the purpose of creating a construction twin, we were also able to explore it in a post-occupancy scenario.

We conducted these four-dimensional scans in our Hub: a flexible work and social space with a wide range of uses and constantly changing furniture layout. Scanning spaces such as this at regular intervals allows us to assemble a detailed timeline of the space. With such post-occupancy twins at our disposal, we can create digital representations of how spaces are used over time.

By combining temporal and spatial information with data from sensors that read environmental conditions and occupancy we can construct intricate models of how people, furnishings and environmental conditions interact to form a comprehensive whole. This, in turn, can help us to operate our premises more efficiently and anticipate how new designs will perform.

By combining temporal and spatial information with data from sensors that read environmental conditions and occupancy we can construct intricate models of how people, furnishings and environmental conditions interact to form a comprehensive whole.

Registration

The scans collected on-site need post-processing. Most scanners on the market are not equipped with the means to store their position within the site and relative to other scans – none of the scans are geolocated in relation to the site or to each other. This means that when all scans are merged, they are not aligned. From here another time-consuming process of aligning the scans ensues. There are a lot of software packages on the market that attempt to automate this process, but it is highly dependent on how well the scan locations were originally planned.

A balance between the number of scans and overlapping coverage needs to be struck. Undertaking more scans than needed leads to longer scanning times on-site, longer processing times and larger file sizes to digitally archive. On the other hand, if fewer scans than needed are taken the process to automatically align them won't find enough overlap and common markers between the scans, resulting in necessary manual intervention to start finding and flagging common planes and landmarks between each pair of scans.

Reporting

The purpose of the scans in the case of construction twins is to compare the point cloud data of the space (i.e., a set of data points in three-dimensional space, representing where the scanner’s laser beam hit an obstacle in space, millions of those points result in a three-dimensional digital reconstruction of the physical space) to the design intent represented in a BIM model. Then, work out and report whether the site is progressing according to plan or not. Laser scanners provide great capturing accuracy for this task, sometimes down to almost two millimetres. But again, in this case, a person needs to sit down and compare both. Yet another time-consuming process that is susceptible to human error.

In the envisioned scenario where such scans will be retrieved on a regular basis, this three-part process becomes extremely inefficient. If the time elapsed between scans is a couple of days, the time between scanning and reporting in this traditional process would become a bottleneck; scans could be collected faster than we could process and make sense of them. So how could Spot resolve, or alleviate, this?

The beginning of the walk is identified by a fiducial marker – similar in shape to QR codes scanned by mobile phones – placed in Spot's line of sight. During this initial walk, Spot will use its cameras and sensors to record landmarks and features in the space to remember the route and "waypoints" it is being driven through. The user could choose to make Spot perform a certain action at set points (action points). Actions vary depending on the payloads mounted on Spot. For example, the robot can use its robotic arms to open a door or move its limbs to stand in a specific pose. In this instance, the most common action used was to initiate a scan and wait for it to finish.

Spot resolves a lot of problems with regards to the scanning stage; the most significant among them being the issue of consistent repetition.

After the initial walk has been recorded, a user could start using Spot's autonomous “auto-walk” feature to replay it. By placing Spot in front of the fiducial marker that marked the beginning of the recorded walk, Spot would prompt the user to pick from recorded missions that started from the same point.

Spot will attempt to follow the recorded path. Even though a construction site is ever-changing, Spot manages to adapt through its capability to bypass obstacles in its set route. If something, or someone, is blocking its path it will attempt to navigate around it, whilst maintaining a safe distance. Recorded actions will be taken at the positions they were previously set at. Meanwhile, a complete blockage will lead to prompts for the operator to choose how to proceed, whether to skip the scan or continue from where Spot is.

Even though a construction site is ever-changing, Spot manages to adapt through his capability to bypass obstacles in its set route.

Spot can store multiple routes, which is a very useful feature. This enables the pre-recording of multiple paths for different zones within a floor plan, or within different projects. The handler can then replay these missions on-demand as many times as they wish. Path planning is as important here as it is in the traditional approach, but Spot's ability to adapt to changes makes it resilient to constant changes on the construction site.

One of the things that can impede Spot’s capability to scan large spaces and complete large missions can be its battery capacity. This is an issue that Boston Dynamics have tried to address by allowing the user to swap batteries without interrupting the robot’s operation and thus not affecting localisation and the scanning workflow. The potential for Spot to charge autonomously via a charging dock is also one of the new features that may help resolve this.

Registration

As opposed to the problems highlighted in performing the registration traditionally, Spot embeds meta-data with each scan it takes. Part of this data indicates the position of the scan calculated in relation to the space. Localisation is computed based on different sources, forward kinematics in relation to the position where Spot was booted, visual and geometric features collected from the space around the robot. Of course, there are also challenges to consider. For example, despite the fact that Spot is adept at dealing with localisation by combining different techniques there are still unresolved issues, particularly concerning some materials (e.g., glass and mirrors) that require the handler’s attention.

Loading the scans digitally results in a great starting point, as all the scans are processed and aligned relative to each other. This is a major advantage over the manual scanning process both in terms of speed and complexity.

Reporting

In and of itself, Spot doesn't offer anything to aid in this step compared to the way it is traditionally executed. Nonetheless, by decreasing the time required for the scanning, it provides more time to focus on reporting and comparing the point cloud data to the BIM models. We didn't want the automation to stop at this point though, and that's why we decided to investigate whether there were any offerings on the market that could aid in this step, before investing time in-house to resolve this problem.

This is how we were introduced to the Avvir Reality Analysis Platform. Avvir delivers the only hardware agnostic platform that leverages a variety of algorithms to – amongst other things – automatically compare point clouds to BIM models and generate progress reports and critical insights. This key asset is the final step to offering an end-to-end automated workflow.

However, it is important to bear in mind that there are challenges for operating robots on-site. These include the ability to maintain correct localisation within the space. Spot attempts to resolve this by combining different techniques, such as localisation based on sensors. Charge capacity and swapping batteries without affecting the workflow is also a problem. Both are challenges that were tackled by Boston Dynamics. Another ongoing issue is on-site safety and maintaining autonomy.