Earth+: On-Board Satellite Imagery Compression Leveraging Historical Earth Observations¶

Due to limited downlink (satellite-to-ground) capacity, over 90% of the images captured by the earth-observation satellites are not downloaded to the ground. To overcome the downlink limitation, we present Earth+, a new on-board satellite imagery compression system that identifies and downloads only changed areas in each image compared to latest on-board reference images of the same location. The key of Earth+ is that it obtains latest on-board reference images by letting the ground stations upload images recently captured by all satellites in the constellation. To our best knowledge, Earth+ is the first system that leverages images across an entire satellite constellation to enable more images to be downloaded to the ground (by better satellite imagery compression). Our evaluation shows that to download images of the same area, Earth+ can reduce the downlink usage by 3.3× compared to state-of-the-art on-board image compression techniques without sacrificing imagery quality or using more resources (downlink, computation or storage).

Introduction¶

Fresh and high-quality satellite imagery is key to many applications, from digital agriculture [29, 56, 72, 73], environmental monitoring [6, 46, 68, 84, 85], to automatic road detection [31, 60, 86], and many more. As a result, large constellations of Low-Earth-Orbit (LEO) earth observation satellites have been deployed [48, 74, 82] to capture high-quality imagery for any location multiple times a day [48, 74, 82].

However, most satellite imagery data captured by these satellites are currently not received on the ground due to the limited downlink (satellite-to-ground) capacity. According to a recent estimate, only 2% of the total image data observed by each satellite can be downloaded to the ground [48]. Some mission-specific satellites handle the downlink-capacity limitation by filtering images onboard the satellite [49, 82] to focus only on mission-specific areas prepaid by the customer. However, this approach is not sufficient for general-purpose satellite constellations (e.g., Sentinel-2 [51], Doves [74]), whose goal is to capture and download satellite imagery over wide geographical regions to serve more applications.

This paper aims to improve onboard compression for satellite imagery. 1 . We are inspired by the observation that the terrestrial content changes slowly between two consecutive satellite visits at the same location [78, 88]. Thus, to compress a new image, we can compare it with a recent image of the same region, called a reference image, to detect the geographic tiles (defined in §3) within the region that has changed and then only compress and download the changed tiles. Our measurement on Planet dataset [74] shows that without the interference of clouds, only 20% of the tiles in each image have changed in the previous five days on average, which ideally can save downlink usage by up-to 5× (§3).

Yet, realizing the reference-based encoding for onboard imagery compression can be challenging because the reference image should be as fresh and contain as little cloud as possible (§3). Typically, the last cloud-free image captured by the same satellite [74] can be over 50 days old on average (§3). With such a large time gap, the reference image and the new image may have substantial differences (more than 50% of the tiles will have significant changes as shown in §3), making reference-based encoding less effective.

We present Earth+, a constellation-wide reference-based encoding system, where the reference images can be selected from historical images of any satellites in the constellation. By broadening the set of potential reference images, Earth+ increases the probability of obtaining fresh and cloud-free reference images. For example, with images from an entire constellation [74], cloud-free images can be obtained every 4.21 days on average, instead of every 50 days with one satellite (§4.1).

Earth+ then leverages the existing uplinks (ground-tosatellite) to upload reference images selected from the whole constellation to the target satellite, as illustrated in Figure 1. (§4.2 will discuss why Earth+ does not leverage inter-satellite links instead.) The key challenge of this design is to handle limited uplink capacity of existing earth observation satellites (e.g., 250kbps [50]).

We present two techniques (§4.3) to reduce the uplink usage of Earth+ without sacrificing the savings on the downlink.

First, Earth+ uploads reference images at a low resolution while still allowing the satellites to detect the most changed tiles (§4.3). The rationale is that low-resolution images are sufficient to decide which tiles have changed, which is easier than quantifying how much each pixel in the tile has changed.

Second, Earth+ does not need to store those unchanged tiles when capturing new imagery, which frees up the storage space. We utilize this freed storage space to cache reference images locally on-board, which allows Earth+ to further reduce the uplink usage by only uploading tiles that have changed relative to the on-board cached reference images.

Besides the two aforementioned techniques, our implementation of Earth+ (§5) also entails techniques to handle satellite-specific issues, including cloud detection, on-board computation constraints, handling different bands of satellite imagery, and bandwidth variations.

To put Earth+’s contribution into perspective, the idea of sharing imagery across satellites in the constellation is not new (e.g., multipath satellite imagery delivery [7, 66]). Earth+, however, is the first that leverages constellation-wide imagery sharing to enable more images to be downloaded to the ground (by better satellite imagery compression).

We evaluate Earth+ on real-world satellite specifications (uplink and storage capacities) of the Doves constellation [25] from Planet Labs. We test Earth+’s compression efficiency on two datasets. The first dataset is collected from Sentinel-2 dataset [51], with 3.6 TB data covering 110 thousand km2 from Washington State. We use this dataset to test Earth+ under a wide range of contents (e.g., mountains, forests, and cities), seasons, and under multiple imagery bands (13 bands in total). Since Sentinel-2 only contains two satellites, we further test Earth+’s performance using the Planet dataset [74], from which we obtain images from 40 satellites for one sampled location (due to the download limit) of 64 km 2 in the U.S. for three months. Our evaluation shows that:

• Compared to the state-of-the-art onboard compression schemes, Earth+ reduces the downlink bandwidth usage by 1.3-3.3× without hurting the imagery quality on all bands. This can reduce the reaction delays of ground applications (e.g., forest-fire alerts) by upto 3×.

• These improvements are achieved without using more uplink bandwidth than currently available or more compute or storage resources than the baselines.

• With more satellites in a constellation, Earth+ can further reduce the amount of downlink bandwidth usage. That said, Earth+’s reference-based encoding is not a good fit for applications that require lossless satellite imagery (§8).

Motivation¶

We start with the background on satellite imagery and earth observation satellite constellations.

2.1 Background¶

Many applications can benefit from frequently updated (e.g., daily) and high-resolution satellite imagery. For example, precision agriculture ideally needs daily access to satellite imagery with each pixel corresponding to a 5m × 5m area on Earth [14, 30] to help timely decisions on the distribution of fertilizers, pesticides, and water. Also, wildfire monitoring requires the imagery to be updated frequently with sufficient resolution to promptly detect and respond to fire outbreaks, mitigating potential damage [82].

To provide fresh, high-resolution satellite imagery, many LEO satellites (e.g., >100 satellites [74]) are deployed to form satellite constellations. Figure 2 shows an illustrative example of a LEO satellite constellation, where multiple satellites are located in a sun-synchronous orbit 2 and these satellites can potentially stream data to the ground when they are close enough to one of the ground stations (we only plot one ground station in the figure for simplicity).

We characterize two features of such LEO satellite constellations:

• High-resolution imagery: LEO satellites are close to the ground (due to their low earth orbits) and can capture imagery with low ground-sampling distance (GSD for short, lower GSD means higher resolution).

• Frequent revisit: With a large number of satellites, any location on the earth’s surface will be frequently revisited (e.g., daily [74]), while a single satellite can only revisit one location once every ten days [36].

Note that in the following text, we denote the ground as the ground stations that the constellation can potentially contact and the computation and networking infrastructures around these ground stations.

2.2 Downlink bottleneck and our objective¶

Downlink capacity gap: Despite more images being captured by the satellites, only a small fraction of data are downloaded to the ground due to the limited capacity of the downlink (satellite-to-ground) [48, 66, 82]. Specifically, we refer to downlink bandwidth as the average download speed from satellites to the ground during each ground contact. The exact gap between the downlink capacity and the imagery data varies with the constellation, and a recent study shows only about 2% of the images captured by satellites are actually downloaded to the ground [48].

Further, the downlink demand is constantly growing, with higher resolution (e.g., a GSD of 0.5m [32]) and more bands found to be useful (e.g., vegetarian red edge band and water vapor band [33]). In contrast, the downlink grows slowly due to the long deployment cycle of satellites. These trends suggest that the gap between the demand for downlink bandwidth and its actual capacity will likely persist if not increase.

Optimization objective: We aim to address the downlink bottleneck of satellite constellations by better satellite image compression. More specifically, we aim to use much less bandwidth to download the same amount of satellite imagery, measured in the number of photoed locations and frequencies, without compromising image quality. To measure the quality of the downloaded images, we use Peak Signal-to-Noise Ratio (PSNR for short), which aligns with satellite imagery compression literature [52, 55, 57, 80].

On-board constraints: While optimizing for the image quality and reducing the downlink consumption, we stick to real-world on-board storage, computation, and uplink constraints. We describe the real-world satellite specification that we used for our evaluation in §6.

2.3 Existing solutions¶

There are several approaches to addressing the downlink bandwidth bottleneck.

Upgrading infrastructures: The first is to physically increase the total downlink capacity of the satellite constellation by upgrading the infrastructure (e.g., building more ground stations [74] or adding more satellites [48, 51, 74]). The costs of such infrastructure changes can be prohibitive, and they can be slow. For example, it takes tens of millions of dollars to build and send just one single satellite [11]. On-board filtering: An alternative is to filter the imagery onboard the satellite [48, 49, 82]. For the mission-specific constellations that focus on specific regions, this approach can filter out most of the imagery. For instance, the Biomass mission targets forest areas to monitor forest coverage changes [3], while the IceBridge mission observes polar ice to gauge climate change impacts [1]. However, they must exclude data useful for other applications. For example, the Biomass mission omits about 91% of the Earth’s surface [10, 13], such as city areas (which are useful for smart city applications) and agriculture areas (useful for digital agriculture).

In-space application processing: Instead of downloading the imagery to the ground, a wide range of systems process the application onboard the satellite and stream the application results back to the ground [2, 15, 16]. However, this approach cannot support many applications due to limited on-board compute, while downloading imagery to the ground allows all applications to perform analytics based on downloaded imagery.

Inter-satellite link for multi-path imagery delivery: Boosted by coherent optical communication [59, 77, 87], the inter-satellite link capacity is quickly growing and allows multipath satellite imagery delivery that can significantly reduce imagery delivery latency [61, 66]. However, this approach does not increase the total downlink capacity of the satellite constellation, or reduce the total amount of imagery data that need to be downloaded, so it is still bottlenecked by limited downlink capacity.

On-board satellite imagery compression: This work focuses on onboard imagery compression, which is complementary to the first three approaches. Existing solutions include augmenting single-image codecs [37–39, 58, 67, 75, 76, 93] and developing more expensive neural-based codecs such as autoencoders [40, 41, 47, 95, 96]. However, these techniques focus on compressing single imagery from a single satellite, so they fall short in leveraging the redundancies between images for higher compression efficiency.

Reference-based encoding¶

Next, we introduce reference-based encoding, a seemingly promising idea that leverages a reference image to pinpoint and download only regions that have recently changed. As we will see, directly applying this approach to a satellite does not work well as images locally available to each satellite may not be recent enough or contain too much cloud to realize the benefit of reference-based encoding.

Background on reference-based encoding: Referencebased encoding is commonly used to compress a sequence of images whose content changes slowly and gradually with respect to time[42, 71, 78, 81, 88, 89], such as video streams. Existing reference-based encoding systems (e.g., video codecs [42, 71, 81, 89]) typically select some of the images as the reference and encode the remaining images by encoding their difference concerning the reference images. As existing codecs encode the images at the granularity of tiles (a tile is a block of pixels, where we use a 64×64 pixel block as a tile by default), and the difference is separately calculated per tile.

Since the satellite imagery captured for the same location also changes slowly over time (as shown in prior work [78, 88]), there is some recent work to apply reference-based encoding in onboard satellite imagery compression [78, 88]. Given a new image, it compares the image with a reference image of the location from the past and pinpoints the changed tiles with a pixel difference greater than the threshold compared to the reference. It then encodes those changed tiles and downloads the tiles in their entirety. 3 Our work follows this approach when encoding changed tiles (§5).

Reference images need to be fresh: While reference-based encoding seems to be a good fit for imagery compression, it is only effective if the age of the reference image—the time gap between the reference image and the currently observed image—is as low as possible. Reference image with high age leads to more changed areas in the currently observed image, which must be downloaded to the ground. Figure 3 provides an illustrative example, where the amount of changes need to be downloaded at Day 30, if using high-age reference images from (Day 1), will be much more compared to using low-age reference image (Day 27). To make it more concrete, we use three months of cloud-free (explained shortly) images from the Planet dataset [51] on one randomly sampled location in the U.S. Here, we say a tile has changed if it has an average pixel differences greater than 0.01 after aligning the illumination (§5). 4 Figure 4 shows a steady increase in the percentage of changed areas with the age of the reference image: the percentage of changed tiles will increase by 3× if increasing the age of the reference image from 10 days to 50 days.

Reference images should be cloud-free: If some tiles in the reference image are covered by clouds, they are not useful as a reference to detect changes. As a consequence, the corresponding tiles in the current image can only be deemed as changed and downloaded to the ground. This greatly compromises the benefit of reference-based encoding.

Why reference-based encoding is challenging? In practice, however, there may not always exist a reference in the satellite’s history images that is both fresh and covered by little cloud. For example, existing work [78, 88] stores a fixed reference image on-board, which will get older over time and make most of the areas being counted as changed and downloaded to the ground, negating the benefit of referencebased encoding. Moreover, even if a satellite were able to choose the reference image from all of its historical images, the most recent reference image with less than 1% cloud coverage would still be tens of days old. For instance, Figure 5 shows the age distribution of the closest reference images that are covered by less than 1% cloud if the satellite chooses the reference image by itself (i.e., the “Satellite-local” curve in the figure). We note that the age of the most recent cloudfree reference image is 51 days on average. The reason for the high ages of recent cloud-free images is two-fold:

• A single satellite revisits the same location at a low frequency (once every 10-15 days [36]). This is because LEO satellites can only capture a small area on Earth at a time (since their size is small [82] and they are close to Earth), necessitating extended periods to complete a full scan of the Earth before revisiting the same locations.

• Since, on average, 2/3 of the earth is covered by clouds [12], so even if the most recent image of the same location is ten days old, it may likely be (partly) covered by cloud and are not ideal choice for reference images.

Earth+: Constellation-wide Reference-based encoding¶

To improve onboard satellite imagery compression, we present Earth+, a reference-based encoding system that obtains fresh and cloud-free reference images from images captured by any satellites in the whole constellation, rather than the history images of the same satellite. This section introduces the idea of constellation-wide reference sharing (§4.1) and an overview of Earth+ (§4.2). We then present the design of Earth+ that makes constellation-wide reference-based encoding practical (§4.3).

4.1 Constellation-wide reference selection¶

Compared to the prior work, which only refers to local images observed by the same satellite, Earth+ augments the set of reference images that reference-based encoding can choose from and thus potentially reduces the age of reference images, leading to fewer changes to be downloaded to the ground.

To illustrate the benefits and challenges of Earth+, we contrast two designs.

• Satellite-local reference: Pick the latest cloud-free image observed by the same satellite as the reference image.

• Constellation-wide reference: Pick the latest cloud-free image observed by any satellite in the whole constellation as the reference image.

Note that the latter is not practical because it needs a large amount of bandwidth to share the reference images, a challenge we will tackle soon in §4.3.

Figure 6 gives an illustrative example of this contrast with a constellation of three satellites (in different colors). The goal is to compress images taken by these satellites for the same location. To simplify the discussion, all images in this example are cloud-free. Each satellite takes a cloud-free image every 30 days, so the satellite-local reference (Figure 6(b)) will be 30 days old. Consequently, in the last three images (Day 31, 41, and 51), 45%-65% of tiles are deemed as changed and need to be downloaded.

In contrast, with constellation-wide reference Figure 6(c)), since the reference image can be from any satellite, the freshest reference is only ten days old rather than 30 days. As a result, two of the three last images do not have any changed tiles and one has only 45% changed tiles, i.e., only 15% are changed tiles on average. In short, the ability to pick reference images from any satellite in the constellation reduces the age of reference images by 3× (30 days to 10 days) compared to the satellite-local design, and this reduces the changed tiles to download by 3.6× (55% area to 15% area).

4.2 Earth+ workflow¶

Earth+ is a concrete design of constellation-wide reference-based encoding. It answers two basic questions: (1) which reference images should be shared between different satellites, and (2) how to share these reference images using the existing infrastructure.

To answer the first question, Earth+ reuses the images downloaded to the ground from all satellites and selectively uploads these images as reference images to the satellites. Figure 1(b) illustrates this workflow.

• During previous ground contact, the ground station uploads latest cloud-free images (that can come from any satellite in the constellation) as reference images for the locations that the satellite will fly by before the next ground contact 6 .

• When passing over a location, the satellite captures the imagery, removes clouds, detects changes using the reference images, and encodes the changes.

• During the next ground contact, the satellite downloads the encoded changes to the ground.

Compared to the workflow of traditional satellite imagery processing pipelines, which capture images and download them to the ground (as depicted in Figure 1(a)), Earth+uploads the reference images from the ground to the satellite. We rely on ground stations as an “overlay” point to share images downloaded from each satellite with other satellites. The rationale is two-fold. First, the ground stations can access any historical image observed by the whole constellation, allowing Earth+ to select reference images constellationwide. Second, the ground station has sufficient computing resources to more accurately detect clouds and upload only cloud-free images to satellites as the reference (§3).

A seemingly promising alternative to enable constellationwide reference is to let satellites share data via inter-satellite links (ISL). Earth+ does not use ISL because it is currently not available for earth observation satellites [82]. Further, the scale of existing earth observation constellations (less than 200 satellites) is insufficient to guarantee a stable ISL connection between any two satellites, as providing such a guarantee typically requires thousands of satellites (e.g., Starlink [69]).

4.3 Tackling limited uplink bandwidth¶

However, using the uplink to upload reference images to the satellites is not without challenges—the uplink has limited bandwidth (e.g., only 250 Kbps in DOVEs constellation [50]). Earth+ tackles this challenge with three ideas. Put together, they allow enough reference images to be sent to the satellites under the limited uplink bandwidth while allowing Earth+ to realize sizable downlink savings.

Downsampling reference images: Earth+ compresses reference images by downsampling (i.e., lowering resolution) and detecting changed tiles at a lower resolution. For example, if the original image is 4000x4000 and the reference image is downsampled to 500x500, the satellite will also downsample the captured image to 500x500 before calculating pixel differences and detecting changes. We then mark the tiles with average pixel difference over a threshold 𝜃 (see §5 for details on how to pick 𝜃) as changed tiles and only encode and download these changed tiles.

Detecting changes with downsampled images is less accurate than with full-resolution images. However, we notice that it mainly triggers false negatives (i.e., changed tiles might be mis-detected as unchanged). This is because the downsampling essentially averages out the pixel changes in a tile, so the amount of changes are lower compared to without downsampling. As a result, Earth+ uses a lower threshold 𝜃 to recall those false negatives.

To evaluate the effect of reference image compression, we compress the reference images using different compression ratios, and lower 𝜃 properly to align the amount of changed tile between different compression ratios (so that the amount of data need to be downloaded is aligned). In Figure 7, we show that we can compress the reference image by 2600× while only missing 1.7% changed tiles.

Incrementally updating reference images: As Earth+ applies reference-based encoding, which does not encode the unchanged areas in the captured satellite imagery, this saves the on-board storage space used for storing captured imagery by about 80% (since 80% of the areas do not need to be encoded on average, as shown in §6) and enables Earth+ to use the following optimization to further reduce the usage of uplink. Concretely, Earth+ locally caches the reference images onboard the satellite for all locations the satellite will visit and only uploads changed areas when uploading a new reference image to the satellite. The overhead of such caching is marginal (about 5% compared to the existing storage space used to store observed satellite imagery 7 ), and thus fits into the storage space conserved by reference-based encoding. Also, caching reference images on-board allows Earth+ to handle occasional uplink disconnection (more details in §5).

Uploading only cloud-free images: Earth+ requires cloudfree reference images to detect terrestrial changes. However, accurately identifying cloud-free imagery on-board can be computationally expensive as it requires neural networks to accurately detect faint clouds [94] and thus not doable on-board. Earth+ thus uses the ground to check if the images are cloud-free retrospectively before uploading it to the satellites.

Implementation¶

Illumination and cloud: In satellite imagery, the time gap between two consecutively-captured images can be hours [74] or days [51]. As a result, two consecutive images in the image sequence can differ a lot in terms of pixel values due to different illumination condition and cloud condition (as illustrated in Figure 8), making the general-purpose change detector (e.g., [45, 63, 81, 89]) no longer suitable for satellite imagery compression.

Note that there are other potential sources (e.g. sensor noise, image misalignment) that can also trigger large pixel differences. Earth+ does not explicitly address them, as they only appear in raw data sensed by the satellite, which is not accessible in public datasets.

Filtering out the cloud: Previous work [48, 78] observes that a wide range of applications (e.g., autonomous road detection, precision agriculture) focus on the geographical content on the ground, allowing cloudy areas to be filtered out without impacting analytic results. Based on this observation, Earth+ runs an on-board cloud detector to identify and filter out clouds. However, as accurate cloud detector is too computationally expensive for on-board use (§4.3), Earth+ runs a lightweight decision-tree-based cloud detector instead, which is a widely used cheap cloud detection algorithm that can still detect and filter out most clouds except for faint clouds and haze [53, 79, 90]. These faint cloud and haze are downloaded to the ground by Earth+ (thus increasing the downlink usage of Earth+) but do not affect the analytic results of applications, as these applications will perform accurate cloud removal as an initial pre-processing step.

On-board change detector: Based on the aforementioned cloud filtering mechanism, Earth+ then uses the following workflow to detect changes. First, Earth+ filters out cloud by detecting highly cloudy areas in the satellite imagery using a decision tree classifier, and remove this part of the data. Second, Earth+ drops those images if more than 50% of the areas are filtered by the cloud filter. Third, Earth+ aligns the illumination between the reference image and the captured image on less-cloudy areas using standard linear regression (since the illumination condition affects the pixel value linearly [92]). At last, Earth+ detects, encodes and downloads changes (details in §4.3).

Encoding changed tiles: Earth+ encodes those changed tiles by selecting the changed tiles as region-of-interest and runs region-of-interest encoding on the whole image using an off-the-shelf JPEG-2000 encoder (Kakadu [19]). While encoding such images, Earth+ makes sure that the bit spent on each encoded tile is a constant 𝛾 by configuring the bitper-pixel parameter of the Kakadu encoder as 𝛾 times the percentage of tiles that are changed.

Choosing parameters for Earth+: Earth+ introduces two parameters: change detection threshold 𝜃 (§4.3) and bit-perpixel 𝛾 . Earth+ chooses 𝜃 by profiling last year’s data on one single location, and uses this parameter on this year’s data for all locations. Earth+ then varies 𝛾 to trade-off between downlink usage and imagery quality.

Handling different bands: Unlike traditional RGB images, satellite imagery typically has multiple bands and the amount of changes of different bands are different. For example, vegetation bands measure the concentration of chlorophyll (which is sensitive to temperature), while traditional RGB bands are less sensitive to temperature. To handle such heterogeneity between bands, Earth+ treats each band separately, which means that Earth+ detects changes band-byband and updates the reference images band-by-band, allowing Earth+ to mark different areas as changed and download different amounts of changes for different bands.

Handling bandwidth fluctuation: To handle uplink fluctuation, as Earth+ locally caches the reference images, Earth+ can randomly skip the updating of some reference images, and instead rely on the cached old reference images (at the cost of downloading more areas). To handle downlink fluctuation, Earth+ leverages the layered codec feature, which allows Earth+ to download less layers when downlink is limited (at the cost of degraded the image quality). The feature of layered codec is widely supported by existing imagery encoders on the satellite (e.g., JPEG-2000 encoders [9, 19]).

Updating reference images: Earth+ needs to constantly update its reference images. A naive design is to constantly patch the reference image with newly observed changes, similar to how a video encoder updates its reference images [81, 89]. However, we found that this approach will gradually degrade reference image quality, since each patch will introduce some artifact to the reference image (which is caused by Earth+’s imperfect illumination alignment due to low reference image resolution) and such artifact accumulates.

As a result, Earth+ instead acquires reference images by whole image downloading: for each location, Earth+ downloads the first cloud-free image observed by any satellite in the constellation. After this, Earth+ will stop whole image downloading for this location for a month. This operation will not introduce large overhead in large-scale LEO constellations, as the overhead of guaranteed downloading is fixed (at most 12 times a year) and will be evenly spread out by all satellites in the constellation.

Note that although Earth+ starts to find a new cloud-free image as reference one month after observing the last reference image, the extra time it takes to actually find such cloud-free image can be excessively long, which indicates that the actual downloading frequency can be much lower than once per month. In the extreme case, assuming that the constellation contains only one satellite, the frequency of whole image downloading will be once every 81 days in average — only around 4 times a year, where this 80-day estimate comes from the fact that Earth+ starts updating the reference after one-month wait, together with an additional wait time of 51 days in average (§3) to actually observe such cloud-free reference image.

Evaluation¶

In this section, we pick two state-of-the-art satellite imagery compression systems as our baseline and evaluate Earth+ against on two satellite imagery datasets. The key takeaway of our evaluation is three-fold:

• Compared to the state-of-the-art onboard compression schemes, Earth+ reduces the downlink bandwidth usage by 1.3-3.3× without hurting the imagery quality on not only RGB bands but also other satellite imagery bands.

• These improvements are achieved without using more uplink bandwidth than currently available, or more compute or storage resources than the baselines.

• With more satellites in a constellation, Earth+ can further reduce the amount of downlink bandwidth usage.

6.1 Experimental setup¶

Dataset: We evaluate Earth+ on two datasets (Table 2 illustrates the details of these two datasets).

Rich-content dataset: We collect 1-year images on 11 geographical locations in Washington State (where each location is of size 1600 km 2 ) from Sentinel-2 dataset [51]. We sample images from Washington State as it contains a wide variety of geographical contexts, including fluvial landscapes, agricultural areas with varied irrigation systems, mountainous regions with large elevation changes, etc, as shown in Figure 9a-e. Since the total file size of this data is 3.6 TB, to handle the large volumn of this dataset, we downsample the images in this dataset by 4×, width and height, where we confirmed on one location that such downsampling does not affect the savings of Earth+.

However, Sentinel-2 dataset [51] only contains two satellites in its constellation. To further show the potential of Earth+’s constellation-wide change-based encoding, we incorporate another dataset with lower coverage but with more satellites available.

Large-constellation dataset: we use Planet dataset [74] that contains multiple satellites in its constellation to showcase the potential of Earth+’s constellation-wide change-based encoding. Due to the quota limit of the Planet dataset, we only sample images on one randomly sampled location in the U.S. (illustrated in Figure 9f), with cloud coverage smaller than 5%. Our sampled dataset contains 48 satellites in total.

Real-world satellite specifications: see Table 1. In this table, we use data from year 2017 to year 2018 as we found the most public satellite specification data during that time period. As a result, such table may not faithfully reflect the specifications of latest satellites.

Uplink and downlink: We use the uplink and downlink specifications from Doves constellation. Specifically:

• Uplink: we assume that the uplink is of 250 kbps [50] and the connection duration is 10 minutes [20, 44]. Here we assume that the uplink bandwidth is a constant, as the uplink leverages the S-band to communicate [8], which is of low frequency, and thus severe weather conditions do not significantly affect its bandwidth [83].

• Downlink: we assume that the ground contact duration is 10 minutes [20, 44] and calculate the average bandwidth required to download a fixed amount of images.

Imagery encoder: We use the off-the-shelf JPEG-2000 encoder called Kakadu [19], which can run on satellite CPU. We note that JPEG-2000 is a variation of JPEG that supports more imagery bands and bit depths and is widely adopted in LEO satellite constellations [51, 74].

Metrics: Earth+ aims to reduce the downlink demand without hurting the quality of downloaded images. We measure the required downlink bandwidth by dividing the amount of downloaded data during one ground contact by the ground contact time (10 minutes [20, 44]) and measure the image quality via Peak Signal-to-Noise Ratio (PSNR for short). This aligns with satellite compression literature [52, 55, 57, 80], and prior work shows that a higher PSNR typically leads to higher application-side performance [43, 62].

We also evaluate the accuracy of vegetation area segmentation on forest areas on one forest location in Sentinel dataset (other locations has low vegetation coverage). We segment the vegetation area by calculating NDVI index [17, 18, 21–24] and thresholding the NDVI index by 0.1 [17]. The accuracy is defined as the percentage of pixels that are correctly identified as vegetation area or non-vegetation area.

Baselines: We consider two state-of-the-art baselines for on-board satellite imagery compression:

• Kodan [48]: drop low-value cloud data and download remaining non-cloudy areas.

• SatRoI [78]: run reference-based encoding using the first image for each location in our dataset as the reference image (we make sure that the first image for each location is cloud-free in our dataset).

• Lossless compression: compress the satellite imagery using lossless compression. We use two codecs that are commonly used on-board: JPEG2K codec (through Kakadu encoder [19]) and CCSDS 122.0-B-1 codec (via TER encoder [34]). The reason that the SatRoI baseline does not update its reference images via uplink is two-fold. First, the current uplink capacity is insufficient to upload even one reference image (even after the default image compression) for each image that will be downloaded during the one-year period of our evaluation. Also, if choosing references from satelliteobserved images, SatRoI may frequently use cloudy images as references (since 2/3 of the earth is covered by clouds [12] and the on-board cloud detector may incorrectly identify cloudy images as cloud-free, as illustrated in §5), whereas we ensure that the reference images in SatRoI are always cloud-free. As a result, our SatRoI baseline performs strictly better than SatRoI that naively updates reference images.

Also, we evaluate Earth+ using the standard JPEG-2000 image encoder, commonly used by existing satellites [5, 28]. While better satellite imagery encoders exist [37, 40, 67, 75], Earth+ complements these works, as these works focus on how to download the imagery in a target area using less bits, and Earth+ focuses on adjusting the target areas so that those unchanged areas are not downloaded to the ground. We also use JPEG-2000 encoder for other baselines.

6.2 Experimental results¶

Single-image compression: A wide range of prior work has focused on single-image compression by augmenting traditional image codecs like JPEG-2000 [37–39, 58, 67, 75, 76, 93] or developing neural-based codec such as autoencoders [40, 41, 47, 95, 96]. Earth+ complements these works, as these works focus on how to download the imagery in a target area using less bits, and Earth+ focuses on adjusting the target areas so that those unchanged areas are not downloaded to the ground.

Change-based encoding: A rich set of literature aims to further compress images by detecting changes between images. A line of work builds video-based codecs (e.g., H.264 [89], H.265 [81], VP8 [42], VP9 [71] and autoencoders [64, 65, 91]) to leverage such redundancy, with the assumption that two consecutive captures have similar pixel values. This is not true for satellite imagery due to varying cloud and illumination conditions. Another line of work [78, 88] develops change-based encoding that is robust to varying cloud and illumination conditions. Earth+ also falls into this category. However, existing work can only update the reference image using single-satellite information, while Earth+ allows updating the reference image using images from all the satellites in the same constellation, resulting in a fresher reference image and, thus, better change-based encoding quality.

In-orbit computing: An alternative way to reduce the total downlink capacity is to have a concrete application in mind and drop out images that are irrelevant to this application (e.g., [48, 49, 82]). However, this approach may drop out images that are crucial for other applications. In contrast, Earth+ only drops areas that are unchanged, allowing Earth+ to be used by a wider range of applications.

Multipath imagery delivering: One can reduce the latency of obtaining newly-observed satellite imagery by enabling multiple satellites to download the same imagery using intersatellite links [61, 66]. However, this approach does not increase the amount of imagery downloaded to the ground, as it does not increase the total downlink capacity of the constellation, or reduce the total amount of imagery data need to be downloaded. In contrast, Earth+ allows more imagery to be downloaded to the ground.

Limitation¶

While Earth+ improves satellite imagery compression, several concerns remain.

Lossy compression: Earth+’s compression is lossy. While it allows downloading more images, lossy compression may not be applicable to applications that require lossless compression. To address this issue, future work can improve the image quality of Earth+ by augmenting uplink bandwidth. Also, one can repurpose Earth+ for lossless compression by performing lossless delta-based compression.

Evaluating on ground-processed imagery: Due to the lack of raw satellite imagery data (i.e., Level-0 imagery data) in public datasets, we evaluate Earth+ on public imagery that is post-processed by the ground, which did not faithfully reflect the impact of geographical misalignment and sensor noise on Earth+. That said, we believe this issue is not severe as Earth+ detects changes using low-resolution reference images, which is less sensitive to misalignment and noises compared to full-resolution reference images.

Control messages: Earth+ uses the uplink bandwidth that is reserved for control messages to upload reference images. That said, we believe this is not a serious practical concern as the bandwidth needed for ground-to-satellite control messages is low (e.g., 2.4 kbps [70]) and do not currently use much of the uplink bandwidth capacity (e.g., 250 kbps [50]). Generalization of results: Our evaluation of Earth+ focuses on a specific set of satellite specs and imagery datasets, but it does not show how effective Earth+ would be if it is used on other or future earth-observation satellites. We hope our work will inspire more research to examine Earth+ in other environments.

Deployment concerns: Though Earth+ only changes software, there may be complications in implementing Earth+ on existing satellites as Earth+ requires a software update on the satellite’s imagery encoding module onboard the satellite.

Stepping back, we acknowledge that Earth+ does increase the system complexity, especially on the ground stations, including sharing downloaded images across ground stations efficiently. However, we believe Earth+ takes the first step towards delivering more images to the ground by constellationwide imagery sharing.

Conclusion¶

While satellite imagery is useful for a wide range of applications, most of the imagery observed by the satellites is not downloaded to the ground due to limited downlink capacity. This work presents Earth+, a new onboard satellite imagery compression system to reduce the downlink bandwidth usage. Earth+ is the first to leverage images across an entire satellite constellation to allow downloading more images to the ground. Earth+ further uses several techniques to judiciously select and upload reference images under limited uplink capacity. We show that Earth+ can compress the imagery by upto 3.3× without compromising imagery quality on all bands or using more computation and storage resources, while staying within real-world uplink constraints.