Saturday, September 2, 2017

Getting Archived NDFD Data

Usually we want to look forward, not backward, but to analyze a past event we may need to look into historic data. We come across this more often than most mariners would as we prepare training materials, looking for good examples. 

Fortunately, these datasets are all stored and we can get to them with normal tools. We look here first at the NDFD data as it is more complex than other data types—weather maps, GFS, RTOFS etc, are much easier. I record this here because I had learned this once, and then forgot it, and had to start over, and I do not want to repeat that.

First, the question is how far back do we need to go.  The real time source provides only the latest run, so this data expires every hour and we need special procedures to look back in time, even if just hours. There may be locations for intermediate data that I do not know about, but for now it seems either get the latest within the hour or wait till it moves to the archives. With maps, on the other hand, we have a full day.  We can get, say, the 06z map anytime until about 09z the next day, when it gets overwritten with the next run. 

Procedure for older NDFD data. 

(1) https://www.ncdc.noaa.gov  This was national climatic data center (NCDC), but is now National Centers for Environmental Information (NCEI).

(2) Click: Access data / Models / Datasets / NDFD / FTP / (There are other access points, but this one seems the most direct:  ftp://nomads.ncdc.noaa.gov/NDFD/

[ If you choose TDS you get to a catalog page https://nomads.ncdc.noaa.gov/thredds/catalog/ndfd/catalog.html  that will load into Panoply as a catalog, which is then easier if you use that wonderful free program. ]

(3) Then select the year and month (ie 201708) / then select the day, ie 20170829, which happens to the newest in the list going back to 2008. Today is 0902, so we note it takes 4 days for data to find its way into the archive.

This is another catalog page, that shows the individual files... all of them, CONUS, Oceanic, Regional etc, and as noted earlier the wind speed and wind direction are in two different files.  There is no pressure, but a lot of other parameters helpful with convection forecasting.

(4) Now we need to know which file to take, and we learn that from the file name structure, which all makes sense (in a sense) based on this page of conventions... except for the 99, which could be 98 or 97; I have yet to decode that part of the file name.

For conus we want the letter U, and for wind direction we want a B. These file names start  YBU

      ybuz9xkwbn   corresponds to  ST.expr/DF.gr2/DC.ndfd/AR.conus/ds.wdir in the live data.

For wind speed we want a C, or files starting with YCU.

     ycuz9xkwbn   corresponds to  ST.expr/DF.gr2/DC.ndfd/AR.conus/ds.wspd in the live data.

Sample files would be:

            File name                                         Start Time
YCUZ97_KWBN_201708292331        2017-08-30T00:00:00Z
YBUZ98_KWBN_201708290016        2017-08-29T01:00:00Z
YCUZ98_KWBN_201708291847        2017-08-29T20:00:00Z
YCUZ98_KWBN_201708291315        2017-08-29T14:00:00Z

In other words, the last 4 digits in the file name is a UTC, and the first forecast of the dataset will be the next whole hour. The only way I presume that is from having opened several of these. 

In files checked so far, 9x=97 on wind direction is about 10 MB with 16 forecasts at 6-hr steps starting at the date, whereas a 98 is about 32 MB with 48 forecasts in 1-hr steps. For wind speed, the difference is 16 MB vs. 47 MB. I have to sort out the 97, 98, 99 significance, and will add it when learned.  I think we always want the 98.

Note the files have no extension, but they will open in LuckGrib as is, or you can add an extension .bin to load them into Expedition. For the latter you can just add both, and then select show both in Weather Set-up to get wind vectors... or just drag them onto the chart and they load automatically. For LuckGrib we need to add the files externally as explained elsewhere and then load them.

I will come back shortly to illustrate this process by recreating the wind, sea, and current conditions at some recent maritime event of interest.



Thursday, August 31, 2017

Wind Speed from Beaufort Force Number

The Beaufort scale is the primary way mariners relate sea state and wind speed. It can also be used as a way to specify wind speeds without reference to the sea state, but this is not used in that manner much in the US. In the UK, however, the MetOffice often gives marine wind forecasts in terms of the Beaufort Force number, according to this standard table, as does the official GMDSS reports from the WMO. (See our Glossary entry on how wind speed got into this system originally.)


The number on the left is called the Beaufort force number (BFN). Used in this manner, it is similar to using feathers on wind barbs on weather maps, which are always ± 2.5 kts. That is, one long and one short feather is nominally 15 kts, but is actually 15 kts ± 2.5 kts. A forecasted 17.5 kt wind would get the same wind barb display as one with 12.5 kts. We do not switch to two long feathers (nominally 20 kts) until 17.6 kts. ( I am not sure how the programs handle exactly 0.5 in this application.... they likely round up.)

The range of wind speeds within a specific BFN varies from 1 to 4 kts, but the average is about 2.5, same as the wind feather spread.

This note is to document a way to estimate wind from the BFN that we have used for years, but it came up in class today that we do not have this written out anywhere.  It can be useful for those of us that are not accustomed to using BFN for windspeed designation.  Furthermore, the wind roses on Pilot charts are often expressed using BFN, in that they show arrows with feathers, and in their system the number of feathers is the BFN, as shown below.


Figure 8.2-5 Section of a U.S. Pilot Chart. August winds near Cuba are given for each 2° of Lat-Lon. In the top right corner, the wind has a 39% probability of being from the east and 32% from the southeast. When the probability is less than 29%, it is represented by the length of the line, which is relative to a printed scale not shown here.
     Predicted wind speeds are in Beaufort Force numbers, with each side of a feather being one number. In the top right the E and SE winds are Force 4 (11 to 16 kts), the much less likely NE wind would be Force 3 (7 to 10 kts).
     The circled numbers are the percentage calms at each location. The current arrows in this chart are marked with speed in kts with a “steadiness” depicted by the line style: a heavy line means 50% steady, thinner lines means 25% to 50% steady, and a dashed current arrow means steadiness is less than 25%. See the steady Gulf Stream near Florida, and the weak unsteady back-stream south of Cuba. Other Pilot Charts do not have the steadiness data and some give the speeds. From Modern Marine Weather, 2nd ed. 

Thus we create a short cut for figuring wind speeds from BFN, namely Wind speed (in kts) = BFN x 6 - 10, or just

kts = 6N-10.

Below are statistics and errors of this method. It does not work for n = 1 (1-3 kts), nor n = 2 (4-6 kts).