Timing and Time Structure in Model Forecast Gribs

Analysis and forecast timing is crucial for good weather work. This is not just a challenge with model forecasts in grib format (the subject at hand), we also face this when using regular graphic weather maps and text forecasts. We cover this in Modern Marine Weather, under the heading "weather map sequencing" near Table 7.2-1. 

The timing involved for the GFS and other global models is very roughly outlined below. (Regional and rapid refresh models have a totally different timing pattern.)

(1) At precisely the synoptic times (HH = 00z, 06z, 12z, and 18z), observers and instruments around the world collect observations and report them via a global network to all the labs around the world running numerical weather prediction models. (A time label "z" means zulu time, which is a nickname for UTC, which was once called GMT.)

(2) The models collect the data and analyze it for quality and then assimilate it into their model programs and start a new global computation. We are now at HH + 2 hr or so

(3) The model completes the global surface analysis and passes it on to other labs. This is now roughly HH + 2.5 hr. Now, or a bit earlier, the model starts computing the various forecasts based on this analysis.

(4) The OPC receives this latest model analysis, which they combine with their own analysis and create the official OPC surface analysis graphic maps and text reports. This is now roughly HH +3.5 hr

(5) The model completes its full extent of forecasts and posts these online for third party access. This is now about HH + 5 hr 10 min. 

(6) Third party providers (Saildocs, Expedition, LuckGrib, XyGrib, etc) complete their download of the analysis and forecasts to their own servers and process it as needed so they can distribute this to their users. This takes us to about HH + 5 to 6 hr depending on several factors, discussed below.

Data are available to us following steps (4) and (6), with actual times depending on the format, product, and source we use, and we have to, of course, keep in mind both local times and dates, as well as the UTC times and dates of the products.  

In the following we first look at the time structure of what we get, and then address the question of when we can actually get it.

What Forecasts Are Available?

When we ask for a set of model forecasts, it is generally assumed that you want the most recent data. Some data sources let us choose between forecasts based on the latest, or forecasts based on a specific synoptic time over the past 24 hr, which may not be the latest. This latter option would only be used in special circumstances, such as comparing with past ASCAT wind data, or for filling in something we missed in the past 24 hr. Those offering this past data generally only go back 24 hr. (If we want even older grib data we have to look into archived or reanalyzed data, which is a topic of its own.)

When we ask for model forecasts such as GFS, we specify the Lat-Lon region, the extent of the forecasts, i.e., out 1 day, 2 day... on up to 16 day for GFS, and we specify the forecast interval, every hour, 3 hr, 6 hr, 12 hr etc. If we are asking for data from Saildocs by email, we must know ahead of time the extent and interval sizes that are available. When using data sources from within a grib viewer, they usually have the intervals and extents listed and we choose from what is available.

What is often not so clear, even when using a grib viewer with selection options is the available intervals typically change with the extent of the forecast, and the options we have depend on the source we are using. If we use the GFS model as an example,  NOAA creates these forecasts every hr for 120 hr (5 days), and then every 3 hr out to 16 days (384 hr), but outside of direct request from NOAA, there is no third-party source for data  over this full frequency range. Each third party provider makes a choice of what they believe best meets their user's needs. Samples are shown in Figure 1.

Figure 1. Sample availability of GFS forecasts.

LuckGrib and Expedition (via its direct NOAA link) are the only sources outside of NOAA direct that offer the 1hr data. XyGrib and those who use it only go out 10 days, but they offer 3 hr steps over that full range. Ten days is certainly more than adequate, except maybe for an initial long look ahead for planning. Generally GFS forecasts are good out 4 days, but then fall off. For longer term forecasts, the Oceanic National Blend of Models is likely best.

When planning a departure time, data every 12 hr could be adequate; when doing an optimum routing, you might want data as frequently as available within your airtime budget—but there are other factors to keep in perspective when making that choice.

The question of when we can actually download or request these various forecasts is another topic, covered below.

Time Format in Grib Forecasts

When we ask our grib viewer or other source for the latest forecast extending out, say, 48 hr with forecasts, say, every 6 hr, for wind, pressure, and rain, we can expect that each of the forecasts we get back will include the same parameters, except for the rain. A generic "rain" request is likely to be given in terms of accumulation, and there will not be any rain data in the first forecast, as none has accumulated at that point. (GRIB is a WMO weather based computer format, and rain is a deceptively complex parameter in that system. See notes in Modern Marine Weather.)

This 48 hr forecast in 6 hr steps will show up as one file containing 9 weather maps. The first is a GFS surface analysis, valid at the synoptic time of the model run, followed (in this example) by 8 GFS forecasts, one for every 6 hr past the synoptic time of the run, out to 48 hr. If the synoptic time of the run was 12z on Mar 13th, the last of the maps would be for 12z on Mar 15th.

These files might be identified in your grib viewer or nav program two ways. All programs will list them by their valid times, such as 18z Mar 14 or 00z Mar 15, but you may also see them identified by the extent of the forecast. In that presentation, the surface analysis is called h0, the 6-hr forecast is called h6, the 12-hr forecast h12, and so on. 

Figure 2. Time structure in a set of grib model forecasts.

In this example, based on a 12z run on Mar 13th, the 18z forecast on Mar 14 would be h30; 00z Mar 15 would be h36. In practice we really need to know both identities, when it is valid and how old is the forecast. Some programs (i.e., Expedition) use h6, h12, etc; others (i.e., LuckGrib) use 6h, 12h, etc for this notation.

Each grib or nav app will have its own unique way to step through these forecasts. There will be a way to go from one forecast to the next, or go to any specific specific forecast. Also, since these are digital data, the programs can interpolate the times. You may only have forecasts for 12z and 18z but you can ask for an interpolated map at say 14z, or even 1437z, which you might want, for example, to compare with the wind measurements from an ASCAT satellite that went by at that time. Or you could want to compare with data in your logbook, or data from a buoy given at some specific time. The programs do not specifically warn you that these intermediate times are interpolated, but they are, just as the wind data itself is interpolated at points between the actual grid points for the forecast resolution.

When Can We Get the Latest Model Forecasts?

This timing is important for a couple reasons. First we want to be sure we are comparing the right forecast with the right observations, and second, we do not want to download a new file and spend the satphone airtime just to get back the same forecasts we got a couple hours ago. Also when racing, you might want to make a decision as soon as possible, which means getting the newest forecast as soon as possible.

The fundamental fact we must live with is, the earliest possible weather analysis we can get is going to be some hours old, so to compare our observations with that analysis means we have to rely on our logbook records that go back to that time... or use saved tables or histograms of our instrument readings. The analysis will always apply to a synoptic time, so it is fundamental to record, one way or another, all weather data at each of the synoptic times. Wind speed, wind direction, and the barometer (and the trends of each) are the main data, but estimated sea state, measured ocean current, state of the rain, and sea surface temp are other parameters that can matter, as well as cloud cover. Air temp might be useful in the midlatitudes to investigate frontal passage.

To compare what we are seeing on the water at the moment, we have to look at an interpolated forecast that applies to the present time, as noted above.

If we assume we are not relying on HF rfax weather map transmissions, which are only available at specific times of day (a huge drawback!), then we are considering now the earliest times we can request the data (with the understanding that we could get it any time after that).

When OPC issues their analysis (Step 4 above), we know that they have the model data in hand and have had time to put all the data together. We can learn exact values of how long it takes them to issue the latest graphic analysis maps from the annotated rfax schedule at OPC, a sample of which is below.

Figure 3. Sample of an rfax schedule showing broadcast (left) and available times (right).

Part 2 (West North Atlantic along the US coast) was broadcast at 2138z on rfax (left-most time), but it was actually available to download direct from NWS via FTPmail or Saildocs at 2112z (right-most time) a bit earlier. Thus this graphic map analysis is 2112 - 1800 = 3 hr 12 min delayed, leading to the typical 3.5 hr delay for these graphic products—it will take about 15 min to request it and get it back if all goes well.

These 18z analysis and forecasts from the GFS model in grib format, however, will not be available on the boat or at home for another couple hours. These 18z GFS forecasts from LuckGrib were available at 2320z and available from XyGrib a bit earlier at 2317z. These were about 5h and 20m delayed, which is typical for global forecasts. You can see the most recent delay times for any model via LuckGrib at luckgrib.com/status. XyGrib delay times for the models they provide are available from a menu link within the program. LuckGrib will warn you if you attempt to download data that are not yet updated, providing the second request is identical to the first one.

A working estimate for data delays of OPC graphic analysis is about 3 to 3.5 hr after the synoptic time, and we get the first grib versions of global models such as GFS in about 5 to 6 hr after the synoptic times.

There are, however, a couple nuances to this latter delay that you might run into if you dive further into the process. If we look into the actual computation times of the various models, which we can see at this NCEP link, we note two things. First the GFS analysis computations are typically done at about HH+3h 22m, but the full range of forecasts out 16 days (384 hr) finishes at about HH + 5h 10m. The  intermediate forecasts are completed at intermediate times.

Thus our grib providers could actually provide the GFS analysis (h0) at about HH + 3.5 hr, but the full range of associated forecast would not be ready for another 3 hr or so. If you ask for a 6-day forecast at, say, HH+4 you would get the latest analysis and maybe a couple of the latest forecasts, but the later forecasts you got would be from the previous model run from HH-6, because they had not been completed by HH+4. 

To avoid that potential confusion LuckGrib, and likely other data providers, do not take the data as it completes, but wait the full 5h 10m to start their downloads and processing. This way users get all the latest forecasts referenced to the same model run at the cost of not getting any at the very earliest time possible. In the case of LuckGrib, the global model delays are about 5 to 6 hr.

Another way to learn the actual times the files are available is to look directly at the NOMADS link where they come from. The times listed are the times these files were posted. These selected forecasts, for example, were based on the 12z synoptic run (.t12z) with 0.25º resolution (0p25):

gfs.t12z.pgrb2.0p25.f000                  16-Mar-2021 15:27  289M  

gfs.t12z.pgrb2.0p25.f120                  16-Mar-2021 16:03  331M  

gfs.t12z.pgrb2.0p25.f240                  16-Mar-2021 16:33  330M  

gfs.t12z.pgrb2.0p25.f384                  16-Mar-2021 17:09  325M  

The 12z analysis (f000) was posted at 1527z; the 5-day (f120) was posted at 1603z; the 10-day (f240) was posted at 1633z, and the longest forecast at 16 days (f384) was done and posted at 1709z. This represents the average of 5h 10m for the full set. We anticipate a new GFS v16 in the near future, which might expand this delay another 10 minutes or so.

Grib Data Distribution is Not Guaranteed

NOAA uses two broad categories to describe the development and distribution of data: Operational and Experimental. The model forecasts we are using are all Operational; indeed a primary source of the data is the NOMADS site, which stands for NOAA Operational Model Archive and Distribution System. So the data are certainly operational, but the distribution (dissemination) of the data is still not guaranteed, as we read in this note at the NOMADS site. We see similar disclaimers at almost all NWS sites. The only "official NWS dissemination systems" are NOAA Weather Radio and data obtained directly from the local forecast offices. In short, the main data we care about is not guaranteed to be available to us, and that does not even account for the third-party processing that we also rely on. 

This means we have to be aware that the model forecasts in grib format and other grib products from  around the world may in rare cases not be complete or may be missing. Speaking with experts who know the details, I learned that NOMADS has been remarkably dependable over the years. A recent, rare snag in the system was addressed promptly and users were kept appraised of the situation at all times.

To me, a way more disconcerting example is the recent, unannounced loss of the ASCAT data in grib format. This has still not been announced, nor a remedy proposed.

Another fringe example is the grib version of the NDFD, which are the official NWS forecasts converted to digital format. These latest data include inputs from numerous local offices and as a result the format and timing in some user selected regions (that might span two offices) are occasionally inconsistent. This is not a global product, so we will cover it when we discuss regional grib files.

The main message is, although the data always look very official and consistent, we might periodically see glitches in the products, such as missing parameters, or fewer forecasts than anticipated. This is all based on computer technology, which is very good, but none is 100% dependable... if we want 100% we should go to NOAA Weather Radio, but we better use a radio that is 100% guaranteed to work right.