Monday, April 17, 2023

ASCAT Data in Grib Format — Gone and Back Again

Update Apr 23, 2023: Check comments at the end. Since this article was first posted, this data has come back, and then gone, and come back again. 

ASCAT is the name of a scatterometer aboard two European satellites, Metop A and B. They provide real near live surface wind observations several times a day to mariners located in any ocean worldwide. They provide the exact data we need as sailors, as if the ocean were covered with anemometers on buoys, but more important they also provide the exact data we need to verify numerical weather models such as GFS or ECMWF. The primary US source online is NOAA's Center for Satellite Applications and Research (STAR). Which shows global wind data, updated hourly in graphic format.

The introduction of scatterometer data to marine navigators in grib format came from an exclusive experimental arrangement between the Ocean Prediction Center (OPC) and the Pacific Northwest communications company Ocens, who was the sole source for this valuable data in grib format from about 2007 to 2009. In exchange for their unique access to the data, they agreed to do the custom programming to show this complex data set. At that time, they were distributing data from a specific US satellite scatterometer program, QuikSCAT, and the Ocens program ended when the QuikSCAT program ended in 2009. 

There followed a period of time with no grib access to scatterometer data, until about 2015 when the OPC made data from the new European ASCAT program available to the public in grib format for the first time. At that time Ocens offered it again, but now there were other sources, namely LuckGrib and Expedition, who both offered data downloads and display. This did indeed take extensive special programming from both of these companies, because even in grib format this data set was not at all like showing a standard wind forecast from, say, GFS.

Then, completely unannounced, in 2018 OPC discontinued the distribution of ASCAT data in grib format. Reasons for this were not clear, but we had informed hopes that the problem would be resolved. But as time passed,  it seems with COVID limiting the workforce and likely other reasons, this simply did not happen. No more ASCAT in grib format.

The digital data were still available, but no longer in grib format. It was available in a more complex format called netCDF. Scientists have viewers that show this data, but mariners typically do not.  We posted an article and video on how to use Panoply, one of those programs, but this was not a practical solution. Thus we were again with no ASCAT data in grib format.

Then in early 2022 LuckGrib took up the challenge to convert the NetCDF to grib format so it could be displayed in the LuckGrib viewer—or once downloaded from LuckGrib it could also be viewed in Expedition. So we were back with access to ASCAT in grib format thanks to LuckGrib.

But in March of 2023, NOAA removed its traditional (OpenDAP) access to the netCDF data. They simply stopped providing this data in the same way they provide all the other weather data they distribute.  In short, anyone, including LuckGrib, who might want to offer this data to mariners is once again faced with a long, complex programming job... that would  be hard to justify, especially since over the past year, the data availability had not been as consistent as it was earlier. Sometimes the data were not updated for 3 to 5 days.

In short, we are back in a mode of no ASCAT data in grib format, and this one is likely stick for a while.

It is true that this grib data have not been used as much as it should be by the overall marine community. We cannot take blame for that. For years we have taught that this is the most valuable way to check a model forecast—we have a long chapter in Modern Marine Weather on ASCAT. But it is definitely not plug and play. It takes effort to coordinate (in both space and time) when useful data are available, especially when we want to overlay the grib versions of both model forecast and an actual ASCAT measurement.

Below is a sample of how nicely and precisely we used to be able to compare forecast and measurement using grib format.

An ASCAT pass of measured surface wind speeds on July 2, 2022, downloaded from and viewed in LuckGrib. The colors are not wind speed, but rather a time scale of relative valid times. The red sector is enlarged below.

In this image from LuckGrib, the colors represent wind speeds. These actual measured winds can be (could have been!) compared with the GFS forecasted winds.  This pass took place at 1151z on July 2, 2022.

Here is the 12z GFS wind forecast for the region shown above. This comparison is within minutes, so should be good. We added the red lines so we can see where we have actual measurements.


The value of the above comparisons in grib format is we can zoom in and look at specific digital values for a precise comparison.... and indeed we see here that the GFS analysis was very good, covering a large span of wind speeds and subtle wind angle changes.  There will be times and places, where the ASCAT reveals weakness in the GFS forecasts.

So that is the type of comparison we can no longer count on.

But all is not lost... we just return to another section of Modern Marine Weather that explains how we can access the graphic presentation of the ASCAT winds with an email request at sea. Then these images can be georeferenced and displayed with the model wind forecasts overlaid on them for the same time of model evaluation. The challenges of setting up the best comparison is still the same, but these graphic ASCAT data remain available.

Programs like Expedition and qtVlm have very convenient ways to georference an image once, and then do a single button click for further updates. We have many of these georeferenced files already created, which takes that first step out of the process. 

We should also not forget  OpenCPN, which like qtVlm is free for both Mac and PC computers. It has  a very good button-click georeferencing display option in its Weather_fax plugin for the Windows version. The general grib parameter display options are not as strong as in other programs, but the key wind comparison needed for this application works well, and its image display function in the Windows version of OpenCPN is one of the best.

Below are samples of overlaying a model forecast onto ASCAT wind measurements in graphic format as a way to evaluate the model forecast. These graphic data have been dependably available for many years.


Overlaying a GFS forecast on the graphic presentation of ASCAT wind measurements using qtVlm. These are the same data shown above, but not at the same locations. The graphic files are about 37 kb so they can be requested by email. The files names and how to get them are discussed in Modern Marine Weather. We also have a video on the process.


We no longer have the digital read out of the ASCAT winds, but with little practice the color bar provided is adequate for the wind speed comparisons, where we have to accept ±2.5 kts as in agreement. Also it is important to note that the convenient grib format had a very long latency time, because it took a long time to process this data once accumulated. The graphic data, on the other hand, is updated much more quickly so we can always compare the latest forecast with an appropriate satellite pass.

Here are the various ways we can view such data in Expedition now and in the past when we had the ASCAT grib data. 

Here we see the ASCAT as grib file overlaid on one of the ASCAT graphic image files from above.


Here we zoom in to the pic above to see that the graphic data and the grib data are the same.


Here we overlay the GFS grib forecast with the grib version of the ASCAT data which was, when we had it, an effective way to check the GFS. See below


Here we zoom in to see that for the most part the model is spot on but there are regions in this complex flow of light air where they do not agree, so if we are routing across this region we can take that into account.


Finally we come to the way we will make this comparison going forward by overlaying the model wind forecast onto the graphic image of ASCAT data. Expedition has a super easy way to georeference these images since they have a prominent graticule in view.


I will try to follow through shortly and show how this can be done in the PC version of OpenCPN. 

We  have an index of ASCAT related info at starpath.com/ascat.

In the meantime, I would like to thank Craig McPheeters, author of LuckGrib, for all the fine work he put into this project over the years. It was all in the right direction.

Nick White, author of Expedition, also played a key role in this unique chapter of marine navigation history that many ocean racing sailors benefited from. Maybe one day down the line we will get this data back from the government... in some form.

The time scale is not clear, but the next generation of electronic charts called S-100 has a component for universal weather distribution called S-41X. This system and indeed all weather and environmental  data under the S-100 format will not use the grib format. They plan to use a more generalized format called HDF5. Thus if all data does indeed end up in HDF, we might get the ASCAT back in that format, which, presumably, by then we would all have nav apps that could read it.

Grib is a unique format developed by the WMO specifically to transmit weather data. But I have been assured by the WMO that they are adhering to S-100 standards when ready, and they are all HDF, not grib. I was informed that if anyone wanted grib in S-100, they can propose a change to S-100, and the next opportunity for that will be in five years. In short, they are killing their baby.

But this will be a while down the line. We might be fairly well into transmigration by then.

1 comment:

David Burch said...

A belated update here, Dec 2, 2024. Expedition and Luckgrib have sometime ago made the direct conversion from NetCDF to grib so that these fundamentally important data are once again available from within the Expedition app or from within Luckgrib.