Feature: Life Gets Easier
Professional Surveyor Magazine - April 2008
Martin Flood and Stephan Landtwing
Improvements in computer hardware and software over the past decade have increased the accessibility of geospatial data while driving a major increase in demand for accurate and timely data. Geospatial data providers are deploying new mapping technologies such as lidar and full digital image capture to address this demand.
However, new and advanced hardware forms only part of the solution. Today's commercial data producers also need to constantly improve their production efficiency to keep pace with demand while maintaining project margins. A key component: transitioning from a workstation-centric, tool-specific production management philosophy to a fully integrated enterprise production environment.
The benefits of this can be seen in the case of the Swieradow lidar mapping project in southwest Poland, near the borders of Germany and the Czech Republic. BSF Swissphoto (Regensdorf and Zurich, Switzerland) and Berlin and Pasewalk (Germany) completed this project for GEOMAR SA (Szczecin, Poland) in the fall of 2007. Tracing its roots to Walter Mittelholzer, the Swiss aviation pioneer who founded Swissair Photo AG in 1931, BSF Swissphoto has a long history of providing map data using the latest technology. Now a member of Swissphoto Group, BSF Swissphoto offers a range of commercial mapping services, including large-area airborne acquisition and processing of geospatial data, aerial photography, lidar, and traditional photogrammetry.
The Swieradow project required BSF Swissphoto to complete an airborne lidar survey of three separate areas. The total area to be surveyed was 955 square kilometers, with the large Swieradow/Karkonosze area constituting the main portion of the project. The lidar system used was an Optech ALTM 3100, capable of operating at 100 kHz and at altitudes to 3,000 meters. BSF Swissphoto has owned and operated this advanced lidar sensor since early 2005 and has completed lidar mapping projects covering over 50,000 square kilometers. For the Swieradow survey flights, the lidar sensor was mounted on a Cessna 206 airplane, and a total of 373 flight lines were flown to cover the area.
While efficient field data collection is a prerequisite for a successful mapping project, the efficiency of the data processing can have an even larger impact on total costs and delivery schedules. BSF Swissphoto's experience with similar large-area projects in the past had already led them to deploy a fully integrated production management system to improve data processing, handle project status tracking, increase production efficiency, and provide effective data management tools. After evaluating possible alternatives, including developing their own in-house software, they selected the GeoCue production management software suite from GeoCue Corporation (Madison, Alabama) as a platform in building an integrated workflow management system.
Mesh with Existing Components
BSF Swissphoto needed a production management system that would work directly with its existing workflows, software tools, and best practices established over years of real-world project experience. For lidar data production, GeoCue was already integrated with Terrasolid's TerraScan software, a key tool used in lidar data classification and editing. GeoCue could also be customized to integrate other production software or BSF Swissphoto's own proprietary scripts and code. GeoCue's extensibility meant that BSF Swissphoto could "wrap" their current workflows in a basic framework environment that could then be customized to their exact requirements. Using GeoCue's integrated environment builder tools, they could tailor their existing lidar workflow to suit the specific needs for the Swieradow project. This would allow them to continue using the same workflow processes and software tools it had in the past, just in a more integrated, managed, and monitored production environment.
Once configured for a specific workflow, GeoCue provides a checklist system that "cues" technicians to the next step in production. Every entity in a project, such as a lidar tile or an orthoimage, has its own checklist. Each checklist specifies the workflow—what needs to be done—for that entity. GeoCue allows the workflow developer to build his or her own custom checklists and specify the programs and actions that occur when the technician executes each step on the list. These actions can include launching third-party software, running the user's own scripts or code, or simply marking a step's status as "in progress" or "complete."
Since GeoCue included checklists that integrated TerraScan "out of the box," BSF Swissphoto could rapidly customize the checklists to the specific workflow required for the Swieradow project. Consequently, they could deploy GeoCue with only minor customization, effectively migrating their established processes to a full distributed multiuser system without having to purchase new workstation tools and retrain their production technicians.
The deployment of the new software at BSF Swissphoto's production office in Regensdorf proved straightforward. GeoCue is a multi-user, client-server application designed as a multi-tiered architecture with "thick" clients running on each production desktop but with project data stored by the centrally located GeoCue server. The server uses a conventional database (Microsoft SQL Server) for the storage of project metadata, with disk-based files used for the storage of data types such as imagery, elevation data, laser point data, and so forth. Installing and configuring GeoCue required the installation of the GeoCue Server and Clients and the setup a data repository or warehouse.
Once configured for the Swieradow project, the software provided an integrated project management environment for BSF Swissphoto's production technicians and project managers. The GeoCue Client was the common interface into the Swieradow project database, displaying project layers, entities, and checklists as required by the production technician for the task at hand, all in a familiar map-oriented graphical user interface. GeoCue provided views of the Swieradow project status to any users logged into the database. When one user performed an operation, such as locking a lidar tile for editing, all client displays were updated to reflect the locked status of the object. All tasks completed in the project were tracked and updated automatically by the GeoCue Server, including generating production logs that recorded time and effort spent on each individual checklist step. These automated production logs are a critical part of analyzing project performance for future workflow improvements. The software also provided controlled access to the project status and project data at both the user and project level.
The actual production workflow for Swieradow followed BSF Swissphoto's established lidar data production processes. After reducing the raw observables and generating an integrated GPS/IMU solution for the aircraft trajectory (smoothed best estimate of trajectory, or SBET), the coordinates of the individual laser points were output from Optech's DASHMap software. After the flight lines were generated, the lidar points were imported into GeoCue and tiled into 4,000 .5 by .5 kilometer working segments. This was necessary to keep the data file sizes manageable. The tiling resulted in an average of one to three million points per block, a size that could be comfortably handled by all subsequent software tools.
The software includes integrated tools for performing this critical data segmentation step, providing an immediate efficiency gain for BSF Swissphoto. It also handled all the file management by linking each working segment in the graphical map display of the project directly to the data file stored in the data repository on the network, eliminating the need for production technicians to constantly verify where project data files are stored (and if they are current). This improves time management and avoids many costly errors caused by technicians incorrectly processing the wrong data file. Simple color-coding in the map view enabled technicians to have an immediate view of where each lidar tile was in the workflow.
Once segmented into working tiles, the lidar points were automatically classified in four unique point classes: ground, features, low/uncertain, and outliers/errors. This was accomplished by using a series of classification algorithms in Terrasolid's TerraScan software. The algorithm parameters were fine-tuned by BSF Swissphoto technicians to yield the best results for the terrain encountered in the Swieradow project area. Typically, this type of automated classification would be done on a single workstation or manually distributed in large blocks across various workstations by the production supervisor.
Engine Improves Control
However, technicians used GeoCue's integrated distributed processing engine, or Command Dispatch System (CDS), to improve overall control and management of these automated processes. Using CDS, technicians created batch jobs of classification macros automatically dispatched to, and distributed across, any available workstations on the network by the server. Technicians had access to desktop tools, including integrated remoting, scheduling, and distributing capabilities, all controllable from the Client GUI.
These tools included:
- scheduling a task to run at a later time on the user's workstation;
- scheduling a task to run now or later on a remote workstation or server;
- splitting a task across multiple machines (distributed processing);
- monitoring the progress of dispatched tasks;
- suspending tasks with the option of rescheduling upon restart; and
- killing tasks that have been dispatched.
BSF Swissphoto has realized that distributed processing is an important aspect of improving overall data production efficiency, not just on lidar projects such as Swieradow. Often, what is practically important in a real-world production shop is not the absolute number of staffor computers, but the ratio of people to machines. This equates to the number of CPUs each person can realistically control to handle automated tasks while continuing to work interactively on their own workstation. This span of manual control is typically no more than two to four machines, and control is usually accomplished by some combination of remote access software (remote desktop), co-locating multiple computers in a single cubicle, and "hot-seating": physically moving from workstation to workstation.
Each of these "distributed processing by user" methods has serious drawbacks. Notably, they all suffer from very limited scalability. Using these methods, it is inefficient for a single person to control 10 or 20 nodes, especially if each needs to interact through a different GUI. It's even harder to efficiently allocate work across these machines based on CPU load or available resources at the exact moment a task is dispatched.
Other limitations include:
- There is no central management or control of all "distributed" process across all machines available on the network.
- Failures during stand-alone batch processing can result in an abrupt conclusion or corruption of the entire batch process.
- The user has no ability to automatically take advantage of idle machines, reconfigure the batch process on-the-fly as new machines become available, or selectively burden power machines with the bulk of any batch processing effort.
- There is no inherent ability to leverage multiple-core, multiple-processor machines.
For the Swieradow project, BSF Swissphoto removed the above limitations by taking advantage of the integrated distributed processing environment to improve efficiency when running automated classification algorithms on the data. In the future, BSF Swissphoto plans to extend these efficiency improvements to other areas of their geospatial data production by extending the workflow management tools to other product areas.
By building an integrated production management environment customized for its own specific needs, BSF Swissphoto completed the Swieradow project on time and under budget, improving their overall production efficiency without sacrificing quality. BSF Swissphoto can take the lessons it has learned on the Swieradow project and make workflow improvements in the future, allowing them to implement a true continuous improvement plan for all its geospatial data production.
About the Authors
Martin Flood, manager of GeoCue in Canada, provides workflow consulting services and serves as the company's domain expert in lidar technology. He received his undergraduate degree in physics from the University of Waterloo and his M.Sc. in physics from the University of Western Ontario.
Stephan Landtwing, project manager in BSF Swissphoto's 3D mapping division, manages all phases of international lidar and related geospatial projects and the company's production processes and environment. He graduated from the Swiss Institute of Technology Zuerich with a Master of Science in geomatics engineering after completing his diploma thesis on lidar accuracy considerations.
» Back to our April 2008 Issue