We are living in the Golden Age of Remote Sensing, with unprecedented and ever-growing access to satellite data about our home planet. Satellite data is baked into decision-making worldwide, informing technological progress, scientific research, business interests, agricultural planning, national security, and more. The government and commercial remote sensing industries have distinct and complementary roles to play in this tapestry. To help us make sense of these dynamics, we spoke to Dr. François G. F. Smith.
Dr. François G. F. Smith worked at Maxar Intelligence for 25 years, where he was awarded the title of Distinguished Member of Technical Staff. At Maxar, he developed algorithms and workflows for the production of satellite-image derived data. He has two patents pending for automating land cover classification. He was the primary scientific developer of the workflows for global land cover, change detection, intermittent water (flooding and soil moisture), satellite-derived bathymetry, tree stage-of-growth, tundra, kelp, and many more global datasets.
He was also involved in image radiometric and geometric calibration and validation, and created a photogrammetric and machine learning solution for correcting image smear distortions caused by orthorectification. He has a Ph.D. in Remote Sensing from the University of South Carolina.
Smith has been using Landsat imagery for nearly 30 years. He has calibrated commercial imagery using Landsat, created land cover change detection algorithms, and mapped crop type and yield both regionally and globally. Many Landsat-derived products, such as the USGS National Land Cover Database (NLCD) and the NOAA Coastal Change Analysis Program (C-CAP) have been essential to his work.
As an avid Landsat user and an expert in commercial satellite data, Smith walked us through how the private and public remote sensing sectors work together.
Responses have been slightly edited and condensed for clarity.
You have worked in the commercial satellite industry for nearly 30 years. What do you see as the value of Landsat to the commercial industry?
The economic value of Landsat is huge for private companies. Industries including utilities, oil and gas, and paper pulp production, among others, have used Landsat image-derived products for monitoring their assets without having to put people on the ground. Agricultural companies and even hedge fund operators use it for predicting crop yields. Landsat represents a major cost savings for many companies.
Because it is free to the public, Landsat imagery allows students to learn and scientists to research many applications. These applications can be converted into marketable products by companies. Entrepreneurs can also get their start in the geospatial industry by building products derived from free imagery.
There is great value to the environment as well since Landsat allows for global monitoring of deforestation, changes in impervious surfaces, changes in canopy density, and other land cover change. By tracking these changes, researchers can estimate changes in carbon storage. These products help leaders to make informed environmental decisions.
Landsat calibration is considered the gold standard for medium resolution satellite data. How does the commercial industry use Landsat as a reference for its own satellites?
Landsat’s superior radiometry is often leveraged for normalizing small-sat imagery. An example of this is Planet Dove imagery. Most small-sat data does not go through rigorous ground-based radiometric normalization because of the cost or inherent image integrity. So, to transform the raw pixels into radiance and reflectance values that can be used consistently from scene to scene, the imagery must be compared to another reference dataset which has been accurately calibrated to sensible reflectance values. Landsat is very good for this role because the radiometric calibration is very precise, the footprints cover large areas, and it is a global survey continually collected at regular intervals. Simple regression or more complex machine learning tools can be used to calibrate the small-sat data to match the closest Landsat scene geographically and temporally. This also helps deal with atmospheric conditions, since you can use the atmospherically-corrected Landsat imagery as a reference.
Spatial accuracy of Landsat is also very good. Although resolution is 30m (Multispectral and 15m panchromatic) and most small-sat data is <5m resolution, Landsat still often has more accurate geolocation than small-sat imagery corrected using satellite ephemeris. So, matching automatically to Landsat’s grid can avoid a small-sat company from having to implement more costly higher-resolution spatial correction.
Reflecting on your own work in remote sensing, why does data quality matter?
If you have heard of the saying “garbage in, garbage out”, it is very relevant to remote sensing based workflows. If the imagery that is fed into a processing chain is of poor quality, then the output will generally also be of poor quality. There seems to be a little more tolerance of poor image quality in Deep Learning-based applications, but any scientific data products such as vegetation indices or traditional classification requires the most precise imagery inputs. This ensures both accurate and consistent outputs scene to scene over both time and space. Data products such as Satellite-Derived Bathymetry (SDB), which uses Landsat imagery, have so little room for error in their input reflectance values that only imagery with the highest precision radiometry can be used.
What role does the commercial remote sensing industry play in satellite-based Earth observation? How can this interdependence be maximized for the mutual benefit of public and private sectors?
High-precision mid-res imagery is perfect for global monitoring of many phenomena on Earth, while commercial imagery is well suited for small-area object level investigations. Some organizations are using Landsat for monitoring change and where change exists, tasking high-resolution imagery for closer investigation. I personally use mid-res data for creating deep temporal stacks of data to classify specific phenomena and use higher-res, less precise imagery for delineating boundaries of phenomena. Then I can populate the delineated polygons with results of the temporal analysis to leverage the strengths of both Landsat and high-res commercial imagery.
I think the commercial users of imagery need both mid-res high-quality imagery and high-res lesser-quality imagery for many applications. Both technologies are required for sustaining and advancing the remote sensing industry.
Where do you think this industry stands in relationship to NASA’s flagship missions?
I think there is a trend to launch as many and as high-resolution imaging systems as possible, even at the expense of lower radiometric quality and limited band wavelengths. I think that trend will continue, as the industry perceives that is what the intelligence community needs and believes that new tools such as Deep Learning have been developed to handle object extraction from those images. The U.S. Government intelligence community drives the specifications of many Earth observing systems because that is traditionally who drives the commercial market.
At some point, I think high-resolution commercial applications will become more prevalent than before and will leverage both the flagship programs such as VIIRS, Landsat, and ICESat-2 and high-res small-sat systems. If the U.S. government, particularly NASA and USGS, can help ensure that the high-res small-sat data are interoperable with the data from the government flagship systems such as Landsat, then it would help intelligence and U.S. federal government applications, as well as commercial application development. That will reduce the amount of processing that private companies have to perform, and thus keep costs down substantially.
This could be accomplished by the U.S. government data purchase contracts having certain requirements such as the imagery must be Analysis Ready Data with very clear specifications. All small-sats should go through the same process particularly regarding geometric correction to the same 3D dataset.
Do you think the commercial satellite industry will ever be a viable substitute for any of NASA’s flagship missions?
No. These flagship systems are too expensive for the private sector to produce on their own, and the companies are generally too short-sighted to build mid-res exquisite systems, even as they leverage the data themselves.
Also the Earth observing imaging companies are not as stable as I would like to see. Maxar was purchased by Advent, a private equity firm. Its leadership was largely replaced with executives from the more general tech industry such as Google, Amazon, Meta, etc. When leadership of a scientific company is not from home-grown scientific staff, a lot of prior institutional knowledge is lost and the products will change to match the expertise of the new leadership. This makes it unlikely that there will be any improvement to image data quality, and may make it difficult to sustain existing image quality.
Currently, Maxar performs annual ground-based radiometric control on its imagery, but that is expensive and requires implementation of new coefficients into the processing pipeline. This kind of process should be required by U.S. government contracts to ensure it will be perpetuated.
How do you think commercial licensed data can and should be used with open science standards?
I think U.S. government-purchased satellite imagery should be made available free of charge to all scientific researchers, nonprofit, federal and local government, and educational institutions. Commercial companies can pay for commercial imagery through resellers. There should be requirements on U.S. government-purchased commercial imagery to be analysis ready, or formatted for general analytical use, to make it easier to use and interoperable with other image datasets. At the same time, access to the raw, unprocessed imagery should also be required.
Can researchers easily compare commercial satellite data quality with NASA flagship data quality?
Most commercial imagery is not directly comparable to Landsat imagery in terms of image quality. There have been efforts in the industry to create standards for image data quality such as the Committee on Earth Observation Satellites (CEOS). And there are interagency groups to share accuracy assessments, such as the Joint Agency Commercial Imagery Evaluation (JACIE), which routinely compares civil and commercial remote sensing data. These efforts are generally led by the U.S. government.
But really, there is no standard that all Earth observing imaging companies are required to adhere to. Generally, “good enough” is a qualitative rather than quantitative determination. Some high-res commercial companies, such as Maxar and Airbus, do perform ground-based and in-flight calibrations to their data which are on par with Landsat imagery. Maxar performs their own calibrations and validations using their own spectrometers and tarps. They collect spectrometer readings at the same time and place as an image is collected, and calculate slope and shift to apply linear regression to calibrate the imagery. Furthermore, they use a process called Acomp to transform the imagery to at-satellite radiance and then compensate for atmospheric contamination to produce the final surface reflectance output. The calibration process was originally worked out by NASA scientists and shared with the community.
The small-sat companies generally do not perform ground-based calibrations, possibly due to poor imagery integrity or the expense of the calibration and atmospheric compensation processes. However, NASA makes calibration and validation much more affordable. For example, NASA’s 6S is a popular radiative transfer model available to the public at no cost, and NASA’s AERONET stations are very useful for assessing how well atmospheric compensation processes have dealt with aerosol contamination.
Regarding image-derived products, “good enough” accuracy is determined by a specific customer or contract.
What is your hope for the future of cooperation between the public and private remote sensing sectors?
I think the Commercial Satellite Data Acquisition (CSDA) program is good for bringing commercial high-resolution data to the scientists which can be too expensive for most researchers to afford themselves.
As new space and satellite technologies are developed by NASA, they should be shared with the commercial sector and thus inform what is possible in terms of satellite remote sensing. Free tools such as deep learning models, calibration tools, and access to the cloud are really helpful for academic and commercial sectors to develop applications from NASA technology.
I hope the government will continue to support commercial satellite companies by purchasing their data and using it for important military, intelligence, and environmental applications. Ultimately, the remote sensing industry is and always has been linked to the military. In the future, hopefully the remote sensing industry will be able to stand on its own a little better commercially, while still serving the military and intelligence communities.
I hope NASA will continue to maintain an unbroken 50+ year database of global Landsat imagery. The Landsat archive is invaluable as a look into the past and allows for extremely deep temporal variables to be created. That is indispensable when understanding how features change over decades such as water extent and glacial and polar ice.
