Obtaining Buoy Data by Email for the Model Accuracy App.

Model Accuracy is a unique program that allows navigators to compare their logged wind and pressure data over a recent past period with one or more model predictions in order to evaluate the model. For typical use underway, the computer does not need to be online (via satcom), because the grib files and wind data are already stored on the computer.

The program also offers a way to compare the models to buoy data, and indeed when the computer is connected to the internet, it has a super convenient way to download the buoy data to make the model comparison. Underway, however, this link does not work unless the user has set up a rather complex interface from program to satcom.

What we want to show here, however, is we can get the needed buoy data underway from Saildocs so that this powerful function of the program is still active underway.

The program needs to have the model GRIB data in one folder (say, MyGribs) and the logged observations or buoy data in another folder, say MyData.

Procedure

Step 0.  

For this method it is best to record some information ahead of time about the buoys you will want to use. Go to NDBC for your buoy of choice and record the lat, lon, and height of anemometer in meters. In this example using buoy 41010 we get:

28.878 N 78.485 W (28°52'39" N 78°29'6" W)
Site elevation: sea level
Air temp height: 3.7 m above site elevation
Anemometer height: 4.1 m above site elevation

Rewrite the red part as:

28.878 -78.485 4.1 

(in decimal lat-lon, w lon is negative). We will use this later.

If you did not get this info ahead of time, then send this query to saildocs:
send  ftp://data.ndbc.noaa.gov/data/latest_obs/41010.txt
and you will get back the latest report, which includes the Lat, Lon of the buoy that we need. Then just assume that the height is 4.0 m.  Most are 3.5 to 5.0

Step 1.
To get the data: send email to query@saildocs.com, with this single-line, plain-text message

send https://www.ndbc.noaa.gov/data/5day2/41010_5day.txt

For two buoys use two lines same but for buoy number.

You will get back a long text file, one for each request, whose top part will look like:

Step 2.
Select the data part leaving off Saildocs headers and footers, then save to a text file and change to courier font to see something like this

Step 3.  
From your records of the buoy data at NDBC, insert at the top left the lat-lon and anemometer height in meters as shown below and then change the file name to

41010_20200302_2108.noaa.txt

This is buoy number_yyyymmdd_hhmm.noaa.txt, where the hhmm is the UTC that you downloaded the buoy data. It will be in the Saildocs header.

So it now looks like:

Step 4.
Save this file into your MyData folder and carry on with Model Accuracy as if you had fetched the data online.

Forecasting and Evaluating Local Weather

For local weather, meaning inland and near coastal, we need regional models. For the day of the sail the workhorse is likely to be the HRRR model, but it goes out only 18 hr for most forecasts.

Here is a summary of the models we would use

The last one listed is not really a model, but the National Digital Forecast Database, which is the digital form of the forecast made by meteorologists of the local Forecast Offices. The HRRR model actually goes out to 36 hr, but only for the runs at the synoptic times (hh=00,06,12,18z), and at least for now, none of the 3rd party providers are offering it. If needed, we can download it manually from here, using instructions from here.  Sources of these hi-res files include Expedition and LuckGrib. Note that some third party sources of NAM and NDFD offer only the 12-km resolutions, which are not often suitable for regional work. If needed, we get the data directly from NOMADS.

One approach is to use the bottom three the night before to be prepared for what to expect on race or sailing day, and then start out with HRRR, updated throughout the day, checking NBM and NDFD periodically.

Checks along the way would be bouy or station reports along your route.... or spot checking the NDFD to see if they have changed the forecast.

As an example we look at an anticipated frontal passage, which is always a prominent event in local waters, because we anticipate a notable veer at the front that can have tactical significance.

Here is a an example from 2020 Toliva Shoals race that took place today in the south end of Puget Sound, which we looked at yesterday late afternoon.

Based on the OPC surface analysis and 24 hr forecast, we anticipated a front crossing the course, possibly during the race.

Top is the analysis at 12z Fri (4 am PST); bottom is the 24h forecast valid Sat 4am PST. Race starts at 930 PST, so there is some chance the front will cross the course.

Further evidence of frontal passage sometime during the day is the local text forecast that showed S wind in the morning and W wind in the evening, and that veer (S to SW or W) is a hallmark of frontal passage.

Both ends of the course show a veer sometime during the day, but we do not know when.  These forecasts are the NDFD data that we can access at marine.weather.gov.  In this display, you can click the map for a forecast at any location.

We want to try to estimate the actual time of frontal passage so we can see how it might enter into the  tactics.  We do this here with the NAM model. A sample is shown below using LuckGrib. This forecast was made Friday afternoon (12z model run) and we advance it to start time at 930 PST Sat.

The NAM forecast for 930 PST Sat made at about 3 pm Fri using the then latest run, which was 18z. So this forecast for a midpoint on the course is represented by the meteogram below

We see in this Luckgrib meteogram that the NAM gives us a very good indication of when the front will come through. Fronts are always a trough, so they will be a local low pressure. In this case the  pressure starts out at just over 1016 at the start (though notably higher earlier in the morning) and reaches a low of just under 1015, after which we see the pressure rise. This is a small drop of just 1 mb but perfectly detectable with a good barometer, which most mariners have in the phone in their pockets! Use our free Marine Barometer app, and read the instructions to get it set right.

And we see the veer, and indeed we see the rain. This is a warm front, so we expect the rain to just slowly increase then stop when the front goes by, which it does.

So even though all the signs are weak ones, this is a textbook example of seeing when a front is forecasted to pass, along with the sharp veer in wind direction it brings.

For comparison in the popular Expedition navigation program, below are the same data presented in the excellent meteogram option from Expedition

Expedition also has a nice option for a table view of the meteogram data, which in turn can be exported as csv.

In the actual race, the winds were notably stronger than forecasted the day before (especially at the southern wind funnel at Nisqually Delta, where it really piped up), and the boat finished the 38 mile course before the front came through. But it must have just missed it. 

Here is the pressure record from the navigator's phone using our baro app.

The pressure drop was slightly larger (2mb) and it looks like it bottomed out a bit earlier than 4 pm but all in all fairly close for such an inland detail, predicted 24 hours out. This also shows how you can use your phone barometer to check the timing of such systems.

_______

Here is a second example, found at the moment from the latest national weather map, of a frontal passage over Lake Superior.  (Not necessarily the best example, as we have so many excellent weather sources dedicated specifically to the Great Lakes.)

This is a section of the 18z analysis 2/15 with a blue line I added showing where the front was depicted on the 24h forecast, valid at 12z 2/16 which was made at 12z 2/15.

The front is predicted to move as shown in 18h, and Lake Superior is right in the middle, so we expect a front crossing the Lake at about 18z 2/15 + 9h = 03z on 2/16.

This is the NAM run from 18z showing the front approaching the west side of the Lake at about 03z and then reaching the other side about 06z as seen below.

In both of these maps we can also see the trough in the isobars that mark the front, but these pics are cut off too much to show it.

The main goal is to show how to use NAM CONUS and the other 3-km models to anticipate frontal passage and other details of interest to sailors.  In each case we also see how the pressure pattern can help us anticipate where we are in the transit based on accurate measured pressures... and the valuable role of a meteogram presentation of the data.

We can also get this from the NOAA presentations of the NDFD data.  A sample is below. I will come back with more details of this source later on.

This is for a second front over the South Sound, viewed from the NOAA weather page, which is pure NDFD. We see the veer and we see the rain, but unfortunately, the NDFD does have pressure.

Viewing NetCDF Weather Files in Panoply

This is the text version. There is a video demo at the end of the note.

We get most of our digital weather data in GRIB format, but sometimes we need to look into special data or more often look into archived data that only exists in NetCDF (.nc) format. Here we outline the procedure for using the free program Panoply. We do so because there are nuances to its use, and the help files are limited. Both Mac and Windows versions are available. They are nearly identical.

We will use a HYCOM global ocean current file as an example. (Global HYCOM is archived, but global RTOFS is not.) When a file is opened in Panoply it looks like this:

The data on the right describes what is in the file, but only parts are in plain language.

(1) To plot the data, highlight dataset water_u, and then click Create Plot. Accept all defaults and click Create.  The plot shows up, but with little information showing, such as below.

(2) Now highlight dataset water_v and click Combine Plot.  The popup shows we will be combining it with dataset water_u. Click Combine.

(3) Now you will see essentially the same plot as before, but now there are two Array tabs. If you like, you can open the tabs to see the data.  Now we are ready to customize the plot.

(4) Create current vectors: Under Array(s) tab, in the top line, Plot [Map] of, change Array 1 - Array 2 to the bottom drop down called Vector Magnitude. You can also check the valid time and date of the data in this Array(s) tab.

(5) Set Scales: click Scales tab. Here is what the default looks like before we start:

5a) Under Units, change m.s-1 to knots, the last in the drop down. 

5b) Under Scale Range, set min = 0, and max = 2.5, and Enter or click somewhere else. Once you choose knots and min = 0, you can choose Fit to Data to see what the largest value is in view as a guide, but generally rely on your own knowledge of what the upper end is. There is also an interplay between this upper end and the tick spacing we address below.

5c) Under Scale Caption, select Custom, then type in "Current Speed (kts)."

5d)  For a cleaner display, in the Tick Format line, change major divisions to 5.

We end up with this panel:

This is now getting closer. Our top title is still wrong and the default vector is for 10.3 kts of current which is just a dot, so we have to fix that.  But first we use the Map tab to focus in on the region we want and more display options.

(6) Under the Map tab, we choose to center the display at Hawaii (20N, 156W) with a region of say, 600 miles in all directions, which we can approximate as 10º for width and height. 

6a) Note W Lon and S Lat are negative, so we enter Lon as -156E in this program. This can all be changed with keystrokes once we get set up.  

6b) Click Fix Proportions because our random choices will not match the projection in use. This correction will also have to be done frequently as we change the view with keystrokes. Once you Fix proportions, the original values we entered will be forced to consistent values. 

6c) This is also the place where you select the Lat-Lon grid and size of the labels. You can play with these to get what you want, including color and size of digits. Below is a sample we just set up where I changed default labels to red at 2º spacing... only to show the option, not that it looks better.

(7) Generally I do not change anything in the Overlays or Shading tabs. There are many graphic overlays to experiment with, and some may be better for near coastal displays, but this is just trial and error to check it out—just remember that the one that is likely best is called MWDB_Coasts_NA_1.cnob. There are also others at the Panoply site to be downloaded.

(8) The contours of fixed current speeds match the divisions shown on the colorbar for speeds, so if you want to change those, go back to the Scales tab to change them. To change thickness and color of the contours, use the Contours tab.

(9) In the Vectors tab, change the Reference Value from 10.3 to 1.0... And now it is looking like current data!

(10) To change the top label, go to the Labels tab, and you can change Title to "HYCOM Currents, 00z  July 15, 2018" for example. There are also several other labeling options.

(11) We look at more on-screen set up below, but there are various ways to output the data, as a saved image or KMZ file for Google Earth. You can also make the picture much bigger using options under Plot in the main menu bar. But it is only from within Panoply that we can digitally click and read actual values.

___________

We are done it setting up the data presentation, now we go to looking at specific regions and reading the data using keyboard commands.

The keystroke commands are not well documented as far as I can tell. Here are the ones I have discerned, mostly by trial and error.  For Windows, change [Cmd] to [Windows key] and change [Opt] to to [Alt].

Remember there is no Cmd Z  or Ctrl Z undo command. Once it's done, you must repair things from the new view... that is why this list of commands is so valuable.

For zooming

Zoom out and center  = Cmd + Opt + Left click (cursor = circled -)

Zoom in and center  = Cmd + Left click  ... or right click   (cursor = circled +)

Regional Drag Zoom = right click and drag or Cmd + Left click and drag (cursor = L)

Panning done by recenter

Center display at cursor = Shift + left click  (cursor = crosshair)

Read data value at cursor

Opt + left click (cursor = ?)

If the picture gets hung up from too fast keystrokes, or you zoom into a few pixels, then try the Map tab and type in say 10º x 10º window size, and Fix Proportions.

Note that with most Blogspot images, you can click to see larger one, then on that one, right click and open in new tab, and then you can usually zoom in farther.

The above notes are in essence an outline for a video demo of this process, which I will link to here as soon as it is done.  Should add one for GRIB files as well.

Note that the format of the numbers on the color bar are determined by a set of programmer's codes in a drop down. Here is the summary of how a couple work.  This comes up in formating the ticks in the Scale tab.

Scale factors:   

the choice " %.xf " means show the value with x decimals, i.e., 

• %.0f shows 1012
• %.1f shows 1012.4
• etc.

%.xG shows exponents with x digits, i.e., x=2, 1.0E+3 and x=3, 1.01E+3.

Once the settings are all as you want them for currents, or even more generally the vector combination of u and v components of any parameter, you can save these as default settings in the menu Plot, Save settings to Preferences.  Then next current file you open it will all be set up.

Another basic point not mentioned above is we have the Interpolate option turned on. This is set with a check box in the top right of the Array tab.  Generally we are safe running in interpolate mode which means the program creates data points in between the actual grid points of the data set. The HYCOM data is on a 1/12º grid,  meaning there is only forecasts every 1/12º of a degree (5 nmi). That is important for interpreting data near shore or around and between islands, but for open ocean this does not matter. The large eddies we see are often 100 to 300 nmi across.

Below is a view with interpolate shut off, which means there is only really 1 data point per square grid.

Notice from the Lat scale that these grids are just smaller than 0.10º square, which makes sense: 1/12 = 0.083º. Do not be misled by the number of arrows present; that is an artifact of the Spacing setting on the Vector tab.  If you did Option + left click you see the same value at each arrowhead with a grid.  Below is the same view with Interpolate turned on.

Now if you to Option + left click you get a different value at every arrowhead.

Video demo of the above exercise.

Estimating Net Current Drift on a Long Ocean Passage

When planning a route across the equatorial current system from northern latitudes down into southern, or vice versa, it can be crucial to be aware of the net current set along the route. You are in trade winds the whole time—other than two hundred miles or so of doldrums between them. The trades will vary from the NE to E in the north and from SE to the E in the south, typically at some 10 to 18 kts. You will be crossing trade wind waves, which are larger than we might guess due to the long fetch and duration of the trades. There will most likely be long stretches of a close reach in these big waves, so it might be hard to maintain a due south course.

On top of that, the net current drift to the west could be large enough that a due south heading could not make the target destination. This note shows how to estimate how much that set will be, so you have a good reminder that one navigational goal, throughout much of the route, will be to make miles to the east whenever you can to overcome the anticipated set and potentially strong SE wind.

Such long range sets can also enter into emergency route selection, and as such we have in our book Emergency Navigation, a section of estimating the net E-W drift on a voyage from San Diego to the Marquesas. In this passage the main set to the west is reduced by a section of the route, about several hundred miles long, where the drift is to the east in the countercurrent, a system we discuss in another article.

The method we used then, which is still feasible for planning, is the use of Pilot Charts as they give climatic current drifts in knots or nmi per day. Pilot Charts are available as PDF from the NGA site (google "MSI NGA" to find the link; MSI is marine safety information), or better still use the digital RNC versions from OpenCPN.org under chart resources.  These can be viewed in any enav program, including of course their own free OpenCPN.

In our online course on Emergency Navigation we have a practice exercise on this process, and this note is essentially an outline of the answer to that exercise (Quiz 9, Exercise 1). This exercise crosses the same basic current system, but instead goes from Hawaii to Tahiti in July, about 2900 nmi on rhumbline 170T, but the punchline is if you sailed that heading the whole way with a speed made good of 6.0 kts you will miss the Society Islands.

To work with with Pilot Charts, you can set up plots like the following, either on printed charts or all done digitally, as this one was.

Pilot Charts for July, N Pac and S Pac. with 3.3º  (~200 nmi) steps marked along the route. 

There will not be current data (shown as green vectors) in each of the boxes where we want it, so we have to the best we can to estimate this based on nearby data. We also need to keep in mind we are working on estimates of the estimates.  We won't know ahead of time what the wind speed is, but generally when the charts specify a range such as 10 to 20 miles a day, we would anticipate the stronger drift with strong trades (approaching or over 20 kts) and the lower values with lighter trades of say below 10 kts.

This then, we leave as an exercise to fill out the list and sum up for a net drift.

Solution by Model Forecasts

Another approach to this analysis is to use ocean model forecasts of the surface currents. We have several options in this part of the world. We can use the global RTOFS model, HYCOM model, OSCAR data, or the French data from Copernicus. We have an overview of ocean currents at www.starpath.com/currents, with these references.

If you are evaluating this in about real time, such as just before departure, then the process is very easy, because the data are contemporary, and easy to come by. You can get the data, usually RTOFS model, from many sources, including Saildocs or LuckGrib, both of which are explained in Modern Marine Weather.

A twist to this solution is the date of the exercise. It is now January, and we want this analysis for July. These currents change somewhat from month to month, as noted in the countercurrent article linked above. An interesting fact about the popular global RTOFS data is that, unlike other model data, it is not archived; so we cannot use that for July of last year. The global HYCOM model,  on the other hand, gives very similar results, and it is archived. [Note: that Atlantic RTOFS is archived from 2011 to Mar 2017 in grib format, but with an unusual format, only visible in Panoply.]

We list sources for all archived weather data in the Archives Atlas of the Weather Trainer program, which is just one of its extensive ongoing resources. Find the archived HYCOM data along with other model data at the NCEI web page (the live ocean data, in contrast, is found on the NOMADS page, or more conveniently from one of the several GRIB viewers, such as LuckGrib).

However, finding the data for this retrospective analysis is not the end of the challenge, because the digital data are not in GRIB format. The are in NetCDF format (.nc), which must be read in a program like Panoply. We have a video on the use of Panoply for viewing archived ASCAT data.

I am not proposing this approach as a standard procedure, but for now, I am just using it to analyze these July currents so they can be compared to the Pilot Chart solution.

Below are the HYCOM currents viewed in Panoply for the route in question. These are the ones I used to read off the currents every 2º going south.

HYCOM currents for July 2019 viewed in Panoply, used to evaluate E-W set to compare with Pilot Chart results for the same study. Keep in  mind, Pilot Charts are climatic values, and the above is one specific year.

Below is the summary of currents, where the E-W component has been pulled out. In other words, when you do this, and you find a current of 1.0 kts to the NW, we have to break that into N component and W components, which for this 45º is just x 0.7.  So 1.0 to the NW adds 0.7 to the West.  For this study we do not care about the N-S components.  The N-S component primarily determines how long it will take, but not the set to the West. Technically it has some effect, in that it slows us down or speeds us up, so the time spent in that current segment is not the same as the 6.0 kts we are assuming. But these currents are small, so that is a relatively smaller effect... i.e., 6.0 vs say 6.4.

Here is first the raw data, followed by a transcription—remember this is the answer to a training exercise.

We are assuming a SMG of 6.0 kts, so time in the current for each 2º of Lat change is 120/6 = 20 hr, then we multiply: 20 hr x the drift to get the set, i.e., 0.4 kts W for 20 hr = net 8 nmi to the west.

HYCOM forecasts for net drift to the west in July 2019. We estimate the missing counter current below which is shown to reduce the set from 252 to 196 nmi.

We ended up in July of 2019 with a total westerly set of 252 nmi, but notice that this year there was no notable counter current forecasted, usually seen in July in the range of about 5N to 10N. That result is not common, and likely unique to this model for this year, so we look into ways to estimate the countercurrent.

Below are the OSCAR currents for this same time period, and they do show the countercurrent in about the right place. It is these data that were the basis of the net drifts shown above.

The OSCAR data shows the same strong patch to the west just below the countercurrent, but it is weaker. The OSCAR currents are 8-day averages of changing values so they are always going to be slower than any instantaneous forecast from another model.  In fact, the estimates above for the counter currents are likely low, but our earlier study showed these as fairly close in this particular system.

Another way we get evidence that there was indeed a countercurrent in July of 2019—and likely to be one when we get there in some July of the future—can be seen in the French ocean currents forecasts from the Copernicus viewer. This data can then be exported to Google Earth, which we have done below.

Ocean current forecasts for July 15, 2019 from the Copernicus program in France, exported to Google Earth

We did not do the current summary from this data, but the countercurrent is clearly visible. We will leave this as part of the net drift exercise to find the total set to the west using this data. It is fairly high resolution in Google earth and the drift speed scale color bar is shown top left in units of meters/sec.  Multiply by 2 (1.93) to get kts.

So the new wording of the exercise is use the Pilot Chart plots given above to find the net drift and then compare that to what you get from the Copernicus forecasts obtained from the links above.

For best understanding of this process, do a Pilot Chart study for the present month using live RTOFS data. Then the digital data is easy to get.