Friday, July 23, 2021

NOAA Discontinues Tide and Current Books — What Do We Do Now?

The short answer for many mariners is: nothing changes—but that takes some explaining.

The official NOAA tide and current books of 1995 to 2020 looked something like these below from one particular printer—but there were several different printers, with different cover designs for the same content.  

Books with this full range of content are no longer supported by NOAA. We may see versions of the four on the left from third party printers this year, but they will not include any of the international data, and the two on the right will not be available. NOAA no longer has that international data to distribute, except for US territories.

The reference station predictions included (now called harmonic stations) would be official NOAA data, indicated by the presence of the NOAA logo on each page, but the subordinate station corrections in Table 2 (if present) would be the 2020 versions, because these correction tables are not supported beyond 2020. The 2020 tide station corrections are likely to be good for 2021, and the current station corrections are likely to be close, but chances are these 2020 Table 2 corrections will become less dependable as time goes by. Recall that Table 2 has always described its corrections as providing only "reasonable approximations of the current at the subordinate stations." The tide heights version includes similar clarifications.

For those who historically obtained their subordinate station data from books like these there will be new procedures or new products called for. NOAA no longer produces tables of corrections, but they do indeed provide the full daily predictions for all stations, harmonic and subordinate at their website

tidesandcurrents.noaa.gov.

This website provides annual predictions for the present year and two years into the future for every station. These are in convenient letter size PDF format, 2 pages (back-to-back) per year. Somewhat like we will be doing in the future with the new NOAA Custom Charts (NCC), we can make our own custom tide and current tables in minutes.

Chances are third party printers will eventually do as the Canadians do now; they print combined tides and currents tables for local regions, rather than entire coasts or oceans. In short, industry needs for printed versions can be created as called for. Furthermore, we do not necessarily need printed documents.

For vessels on longer voyages, we note that tide and current predictions are not required by the USCG to be in print. They can be in PDF format (or text or html) in your computer, tablet, or phone. You can, for example, download the stations you care about from within your phone and save them in your favorite ebook reader library. They handle PDFs very nicely and keep them organized and easy to access.

That is the guaranteed way to have the right data—download the PDFs from NOAA. But there are many apps that provide this data without doing anything. They are using harmonic constants to compute the predictions. These apps and their harmonic constants are not individually sanctioned by the USCG or NOAA. The USCG requirement is you must be using the latest data that comes from NOAA. This then becomes a bit tricky. None of the free phone apps that I know of actually promise that. Most mariners have taken it on faith that a nav app will generate the right predictions, but sadly we can show that is not always the case. 

In the end, if we are going to rely on a nav app for this data, we need to at least spot check a few cases with the official data at NOAA. 

There are instructions at the NOAA site, but since they have so much data there, these steps might be helpful. We start by using their service to find the closest station to where we want the predictions:

1) Go to tidesandcurrents.noaa.gov.  Click your state of interest

2) In the top left popup menu click Advanced

3) In Data Type dropdown, select Tide Predictions or Current Predictions

4) Then zoom the map with a mouse roller or the +/- keys bottom right until the stations show up.

5) You will see harmonic (purple) and subordinate (white) stations, but this distinction has no significance at this point.

6) Click your station to get the data, which will be the present values

7) For an annual set of predictions for that station, click More Data, then for tides, on the right, click Annual Tide Tables; for currents, click the tab Annual Prediction Tables. 

8) For both tides and currents, you have a choice for the times: either GMT (meaning UTC), or LST, which would be the same as the old, printed tables that did not correct for daylight saving time, or you can now choose LST/LDT, which means the times given will correct for daylight saving time when the clocks change, 02 local time on Mar 14 and Nov 7.


The sample above uses local standard time, which does not correct for local daylight time. This is the format that all the old tables had, so we always had to be very careful about that.

Now we have an option to show this data corrected. You will notice below that they are the same up till March 14 and then they show up with an hour added. Historically we had to do that ourselves.


That is an outline of getting the predictions, which is just part of the suggested new training we need on tides and currents predictions. Please see the full list of What do we teach now, at the end of the earlier article.

Commercial chart navigation programs such as Coastal Explorer and Time Zero and others have very convenient and versatile tide and current interfaces, and I am sure they update the harmonic constants from NOAA to keep the US data in sync with NOAA.

Popular free navigation programs leave it up to us to be sure we are using the best harmonics for the predictions, which is a bit of a longer story. The USCG requirement, which matches our own obvious goal to have the best possible data, is whatever we use should be traceable to the latest NOAA data. 

The easy check on this is just use your electronic solution to get tide and current predictions at a place of interest, then get the same predictions from the NOAA link above and compare them. These checks take just minutes.

Looking ahead, NOAA publishes the harmonic values for each station that programmers  can extract and use, but these data are not yet in a form that individual mariners might use to update their own navigation programs. In the future this might change, in which case we could end up with NOAA version dates on our electronic tide and current predictions just as we have on our nautical charts.



Friday, July 16, 2021

No More Tide and Currents Table 2 — Navigation Students Celebrate!

Prior to 2021, NOAA's  Center for Operational Oceanographic Products and Services (CO-OPS) supported the publication of their tide and current predictions in these six books, which we can still find in full online.

Tide Tables

2020 East Coast of North and South America Including Greenland

2020 Europe and West Coast of Africa Including the Mediterranean Sea

2020 Central and Western Pacific Ocean and Indian Ocean

2020 West Coast of North and South America Including the Hawaiian Islands

Tidal Current Tables

2020 Atlantic Coast of North America

2020 Pacific Coast of North America and Asia.

Prior to 2021, these were "the official sources." All other third-party printed or electronic  presentations of  US tide and current data, readily found along the waterways and cybersphere, were derived from these, sometimes mixing up actual locations or confusing standard times and daylight times. 

Because of the potential errors in third-party reproductions, responsible navigation schools always relied upon these primary official sources and taught how to use them. The USCG does the same; all deck license candidates must learn to use these tables, and they are included on the license exams.

Each of these six books listed above (both Tides and Currents) present in a Table 1 the daily predictions at several dozen reference stations distributed along the area covered, and then in a separate Table 2 they list the corrections needed for the hundreds of subordinate stations included.

To find the values at a subordinate station, we would learn from Table 2 what reference station applies, look up the tides or currents for the time range of interest at that reference station, and then we would apply to those values the corrections we get from Table 2 for the subordinate station we care about. For tides, we correct both high and low water times and heights. For the currents, we correct for the times of peak flow and slack, and we correct the speeds and directions of both the flood and ebb. 

The process is readily carried out once learned, but the use of Tables 2, covered on every final exam, has been the bane of quite a few students—some corrections are additive; some are multiplicative; in the time column "min." means minimum, not minutes; the time zone is given in code; corrections can depend on the sequence; and so on. 

But let me stop here to get to the good news before getting to implications of this news. 

The good news

As of 2021, NOAA has done away with Table 2 completely, for both tides and currents. Indeed, they have done away with supporting the production of any book or PDF file that contains more than one station, so there is no need for a table of subordinate station corrections. We can all now just take a slow, deep, zazen breath, and let all of those long hours of hard work on Tables 2 gently slip out of our existence and into space.

Those new to navigation from now on do not have to learn these tables at all. If Table 2 comes up in class, you could raise your hand and say, "Those tables don't exist; why are we still covering this?"

Tide-and-current-wise, we move into a new era, although we have in fact been in this era for over six months now as I write this. 

In a sense, this change should not be a surprise. Here is the Announcement that appeared in all of the 2020 tide and current tables (the six cited above), as well as various Local Notices to Mariners:


In another sense, however, which might account for why I believe very few navigators and even navigation instructors are not aware of this yet, is even seeing the Announcement we might not get the full implication of what is being said. 

Navigation instructors know, for example, that NOAA has not actually printed any tide or current books since 1995. We have for years thought of these as commercial books available from several competing publishers. We have just assumed—or read on the covers—that these publishers got the data from NOAA, and that they were somehow vetted to provide quality products.

What the announcement means by "ending the production of the printed tables" is starting in 2021, NOAA is no longer providing these commercial companies with a PDF file of the book content, plus, even more important, they are ending support and updates of the famous Tables 2, both tide and current. The implication is that the six books listed above will no longer be available from any publisher starting in 2021. 

One snag to getting this important message across is, for example, I have sitting on my desk at the moment brand new copies of the 2021 versions of two of those no-longer-existing books!— or at least it looks like the same books, one for tides, one for currents. The format and covers are identical to last years, and at first glance the titles are the same, but now that I know to look carefully, I see that the titles are not identical. The 2021 versions refer only to North America, with no reference to other continents. Looking inside, we see on page 1 that this is the "Commercial edition," no reference to being an official publication.

This publisher has gone to a lot of extra work to meet the needs of many mariners by creating a custom edition, which offers a stop-gap solution during this transition. But it would not be difficult for a consumer to overlook the subtle differences in the book titles, especially since the vast majority of mariners who historically thought they were required to have this book never used the international data.

Another snag to being aware that things have changed come from our interaction with the latest 2021 electronic navigational chart.  If I do a cursor pick on any of the tidal stream objects (TS_FEB) on the latest 2021 electronic navigational chart (ENC)—these charts are updated weekly—I get back a report of two attributes for average speed (CURVEL) and average direction (ORIENT) which present data direct from Table 2. But more to the point, each tidal stream report (in West Coast waters) tells us to get details from a book called Tidal Current Tables, Pacific Coast of North America, which we would have to assume exists. This statement will likely change to get details from www.tidesandcurrents.noaa.gov.  Some navigation programs could present it as a live link. 

The ENC chart makers at NOAA are aware of this now, and they are quick on their feet with weekly updates. This will likely be fixed in a couple weeks.

And still another reason that responsible navigators might not be aware of this big change is the recent update to Bowditch's American Practical Navigator, NGA Pub. No. 9the bible of American navigation. The 2019 edition was just updated this week, called the June, 2021 edition. This venerable publication seems to have also missed the Announcement or misinterpreted it. They cover Table 2 in this update just as they have for decades. Now, in fact, they could save about 4 full pages of the 764 pages of Volume 1 by removing the coverage of Table 2—or by moving it to the history of navigation section.

Despite these snags to being alerted to the change (which will soon be resolved) there are perfectly logical reasons for this inevitable decision from NOAA, and they outline these in the Announcement. 

Reason (a) the USCG now allows digital copies of tide and current data to meet carriage requirements.

It is not given in the announcement, but the official USCG ruling on carriage requirements  was dated May 21, 2020, which refers to "publications required by 33 CFR 164," among others, that are now allowed to be digital, shown below—with emphasis that they are only needed for the "area being transited," which means mariners do not need all of the stations that were in the old books.

Here is another statement from 46 CFR 28.225

The official USCG wording of the change is in their document: USCG Navigation and Vessel Inspection Circular (NVIC 01-16 CH-2).

The timing of the carriage requirements change (May, 2020) brings to mind something else. This came right at the onset of the COVID pandemic. It was not the worst point, which was yet to come, but when this significant ruling came into law, nine populous states had been in full lockdown for two months. Navigation schools and all maritime training facilities were closed, their instructors off doing something else. These schools have just opened up in the past months. In short, I see quite a few ways to understand how this could go unnoticed. 

Reason (b) from the Announcement is the existence of "online sources," their own of course being the primary one, and "electronic services" that provide the tide and current data. That latter (rather pregnant) term presumably means apps that use the official NOAA harmonics (or, more likely, a verified version of them) to compute the data for any time or station. 

In other words, from now on, we are going to have to get tide and current data like we actually get the data already... or at least like we get it most of the time: we use the tide or current icon in our navigation program or an app in our phone. For crucial inland transits, ships can in some waterways even access high accuracy real-time measured water levels online.

But it is not all good news...

No more international data from NOAA

What is not mentioned in the announcement is NOAA is no longer providing in any format tide and current data from other nations. A check of the Tables of Contents of the six books listed above shows that NOAA did indeed for years provide fairly thorough tide height coverage across the globe as well as notable tidal current data. 

That has all been discontinued, but for a different reason, essentially coincidental in timing.  As I understand it, in the past, NOS/NOAA has had agreements with various nations to reproduce tide and current data from their hydrographic offices together with our own in a book-only format that would be helpful to their mariners as well as ours. That went on for many years, but in each case the other nations were providing NOAA only the annual values of times and heights of highs and lows, plus peak times and speeds for currents. They did not share the harmonic constants that they used to compute the data. NOAA could publish the annual data as given to them, but could not compute it.

Over the years, several of the key suppliers of that data have since changed their policies on this and now treat their harmonics as proprietary intellectual property, and have set up commercial enterprises to sell their hydrographic (and meteorological) data as a way to help support their agencies. Some agencies, notably the UKHO/BA have made further arrangements with these other nations, so they now broker the sales of this data for multiple nations.

The result is easy to explain. If we want official tide or current data from another nation, we have to buy it from that nation or one of its representatives. Any third party source would have to be carefully checked, as it would not be guaranteed. Ironically, these (mostly invariant) international harmonics were at one time public domain, but once withdrawn, they have been aggressively protected against public distribution.


Comparing the old and the new

The main impact of this transition in my mind is not about how we actually get the data underway—we have used these "modern" methods for over 20 years—but rather, the effect this has on teaching this important aspect of marine navigation. It is not entirely unlike the transition from traditional piloting to the reliance on GPS. We want to use the fastest, most accurate method, but once it turns into a black box, we have to devise ways to check it.

Wrong tides or wrong currents in some circumstances can lead to a major screw-up, and indeed one that would be hard to get forgiven. Especially the tides, since they are probably the best predicted environmental parameter on earth, and they are typically uniform over large areas. Currents, on the other hand, are honestly hard to predict, even with the best tools, once you get just a 100 yards or so from the reporting stations, but we at least have to start out with correct data at the stations.

We get insight into what is involved by taking a short last look at the Tables 2. Samples of Table 2 for currents on the West Coast relative to reference station San Juan Channel are shown below for the year 2020, then one two years earlier, and then one 10 years earlier.






The example we have been looking at in other studies is Wasp Passage, and even though we see that the stations have been renumbered we can track this one to note that the correction values do indeed change notably with time. It looks like they were constant for 8 or 9 years (or at least not updated) and then got updated in 2019. A check in the 2020 tables tells us that the San Juan Channel reference station was in fact updated in 2019. It looks like that implies the subordinate corrections get updated as well. 

(Note we can see in the samples above the improvements in Table 2 current corrections. Prior to the update reflected in the 2020 Table 2 corrections, currents were treated as pure reversing currents, so the minimum speeds before flood or ebb were considered zero, or unknown (--). With better data, we learn that these currents are actually rotating, not just reversing. This means the track of current directions is not just back and forth along a line, but rather the direction follows the path of a narrow ellipse. The water is never still, but just at a minimum between cycles, which is now listed.)  

Looking now at the Table 2 for tide corrections for 2020 and 2010 in about the same location using tidal reference station Port Townsend, we see a quite different result. 


It appears that these corrections for the tides are not changing. But more important than not changing over 10 years, these old Table 2 tide corrections still get us the right times and heights of the tides in 2021—at least in the couple cases we have checked, one of which is shown below... which leads us in part to our new approach to tides and currents.


The left is the July 16, 2021 data from the reference station, then the Table 2 corrections are applied—you can think of them as from 2010 or 2020—to get the Orcas tides, and then we look at the correct values from CO-OPS, and then we look at one example of what the Announcement referred to as an "electronic service," namely the tide height function within the navigation program called qtVlm

This specific navigation program provides the functionality to read and display tide and current data, but they do not provide any actual harmonic constants, specifically because of the copyright issues mentioned above. This is a French program, so it has many users without direct interest in the free harmonics for US waters. I then manually loaded a set of harmonics that are described as based on the latest US data and very specifically states that they do not contain any copyrighted data. The set cited in the figure includes both tides and currents. The format of the file (TCD) is called tide and current database. qtVlm will read this format directly. Not all nav programs will. 

(I might note in passing that most navigation apps produced in the US do, in contrast, arrive with harmonics already installed. Chances are these are the latest versions and that they get updated as needed, but it would still be a good exercise to compare a couple subordinate stations with the CO-OPS values.) 

We see immediately two things: the Table 2 data for tides seems to have been constant and accurate over the past 10 years (again, we have done just a couple checks) and we see that the latest xtide presentation of the US harmonics are accurate for the two stations we checked.  That is no guarantee at all that others are right. We have, in a sense, checked just two stations; in the end we have to check all stations we count on. 

But the point here is, with this new CO-OPS approach to the data, we do not have to use Table 2 to check this. We would just go to their website and download the year's data for the actual stations we care about (reference stations or subordinate stations) and then email them to ourselves, and then open them in some ebook reader in our phones. Then we can easily spot check a few neaps and springs in a matter of minutes, anytime we care to. Again, in the end we want to use the apps in our nav program because they are so convenient. We just have to check them first.

The situation with the currents is different, which shows why we should be happy that Table 2 is gone. Below is a sample worked out for Table 2 solutions for currents:


We get the July 16, 2021 reference station data from CO-OPS on the left, apply the corrections from Table 2 (shown above) and get the Wasp Passage Table 2 predictions that way, using the last and most recent values from 2020. Then we look up what the Wasp Passage currents really were on this day from CO-OPS, and they do not agree. Off by about 0.5 kts in one case. The times are off  up to 20 minutes, but that is essentially within the uncertainty we might expect, but speed errors are larger than we would expect. Had we used 2010 values the speeds would have been off a few tenths more. Likewise, if we had used 2021 values, which we do not know, then we would presumably have done better. In short, these current corrections in Table 2 are drifting, and since they are no longer being maintained and updated by NOAA, we cannot count on using them. The same is true with the tides. Even though we do not at the moment see effects on tide predictions, this might be related to update schedules. NOAA is telling us not to use them.

But again, that is all history. Looking at the new methods we must use, we see from the qtVlm agreement with CO-OPS that the harmonics used seem good. NOAA actually tells us the numerical value of the harmonics, 37 sets of 3 numbers for each station, but it is mathematically difficult to get into some formats of the harmonics files to check these digitally. For the most part, the checking must be done by looking at the end products.

Looking ahead, it seems we might be able to use new features of the NOAA website to download a full set of the latest US harmonics for tides and currents in one file. This could then be used in popular navigation programs, hopefully with a date label.  We are looking into this.

Also to note, NOAA no longer uses the name reference stations. These are now called harmonic stations, although this terminology change is not yet reflected in their excellent online Tide and Current Glossary. The subordinate station name has not changed, and the meanings have not changed. They have harmonics for harmonic stations and calculate predictions directly using them, whereas subordinate station predictions are based on a nearby harmonic station by applying corrections. The big difference is they no longer publish those corrections; they just present the corrected data online.


What do we teach from now on?

• First we abandon both Table 2s as a means of finding tides or currents, and we remove any training materials we have on them, and we remove any quiz or exam questions that use them.

• We still have to teach standard tide and current table format. That has not changed, but now subordinate and reference stations are the same. Instructors might be able to reword exercises that used subordinate stations to now use the reference station data that was already in their materials. Developing new exercises from scratch, we can now use a favored subordinate station showing daily data.

• We encourage everyone to go to the historic tide data page and download a set of all six of the 2020 books. We want these for several reasons. Each book has extensive notes on tides and currents in general with several valuable related tables, although much of this auxiliary material will be redundant. It is also of interest to look at the footnotes to Table 2 stations you care about, even though we will not be using Table 2. The Explanations of the Tables 2 are also valuable information, even though we do not use the corrections.

• Also, as we show elsewhere and have briefly outlined above, we can use this historic data to test harmonics not just in US waters but over much of the globe. If the harmonics are good, they will be good for many years.

• Then we teach students how to access the online tide and current prediction pages at tidesandcurrents.noaa.gov. It is not quite as direct from the main page as we might hope since they are presenting users with a lot of other data on this page. Here are now the direct links to the "official data." Both links also include a national map of the station locations.

Tide Predictions by Station

Tidal Current Predictions 

• Then we practice going to specific stations we care about and find how to make an annual PDF table of the data we want. This is fairly transparent, but coaching helps. Below is a sample page of a PDF file.

 


 • Next we might go over various ways to get this official info into a mobile device and the various ways to organize it, especially since we will now be using multiple individual stations. Most phones will open a PDF directly, but with multiple files, we are likely best storing them in an ebook reader library for easy organization and access. NOAA policy now is that to be considered official data it must have the logos on the tables.

Official NOAA current data in a phone, with zoomed view on the right.

 • Then we must teach and demonstrate the importance of verifying any tide or current app they are using. It will likely be easy to show that an app in some student's pocket at the moment has outdated data in it. We will post follow up notes on this statement.

• In short, the tides and currents predictions sections of our courses have to change.

For our own Starpath courses, we have added errata online that renders about five pages of our own textbook (Inland and Coastal Navigation) outdated; we are fixing this now. We have also removed all questions in our online course quizzes and exams that require use of Table 2 in tides or currents. 

I can imagine students in navigation courses around the country saying goodbye to Tables 2 with a party in the coffee room—unless they are preparing for a USCG exam. In that case, the path of least resistance takes you right back to the 1980s Tables 2 in your study materials. Eventually this news will float up to the top and the related questions removed, but this may take a while.

______

Essentially all of the information presented here is available in the old tide and current books discussed above or on the extensive tidesandcurrents.noaa.gov website. However, with that said, I want to add that over the years we have found that the customer support team at CO-OPS have been exceptional in providing prompt, detailed answers to any of our questions, and we remain grateful for that.



Thursday, July 8, 2021

Notes on Bank, Cushion, and Squat

This post is just the annotated graphics that we used in our video about the grounding of the Ever Given in the Suez Canal.



To understand bank and cushion:


• Think of ship stationary and river of water running by it.

• Bow shape at the waterline funnels water into the near shore side, which piles up as it cannot compress.


• This hydraulic head forces water aft and bow out. 

• Since it cannot compress, the water must move faster toward the narrowing gap aft. 

Area 1 x Speed 1 = Area 2 x Speed 2 


Then we have new physics entering into the situation:


• The Bernoulli principle (based on conservation of energy) dictates that the relative pressure in the water will decrease when the fluid speeds up (see lab demo below). This is relative to the pressure on the other side of the vessel. Arrows shown are the differentials.





• The net effect is the bow is pushed out as the stern is sucked in toward the bank.

• Helmsman must turn toward the bank to track straight. 





Reference: https://towmasters.files.wordpress.com/2009/05/squat-interaction-maneuvering.pdf





Tuesday, July 6, 2021

US Coast Pilot Interactive Index

Coast Pilots are crucial references for safe, efficient navigation in US inland and coastal waters. Broadly speaking, they are defined as presenting the crucial navigation information that cannot or is not covered on nautical charts, including important information on weather and currents. We are fortunate in the US that these documents, like our charts, are free publications, both of which are updated weekly.

To encourage the use of these books, we have made a custom Interactive Index to the Volume Coverage of the US Coast Pilots, in part because this has changed not too long ago, and also because the detail we include makes it easier to discern the distinctions at the boundaries of coverage.


We recommend that all mariners download the volume that covers their own local waters and skim through the book to see what all is there. It is almost guaranteed that you will learn something valuable about your own waterway that you did not know before. You will also find out how to reach any maritime related agency you can imagine you might ever need.


The General Information section (Chapter 1) of any volume is a mini lesson in modern chart reading and navigation practice, along with radio usage, and basic weather resources.


In the Pacific Northwest waters, for example, we learn things about the current flow that we cannot learn from the Current Tables alone, because there are not current stations in all the crucial places, or the nearest stations are in much wider parts of the waterway so they underestimate the peak current speeds in the nearby channels... but the Coast Pilot tells us what the channel currents really are.



You can download this interactive PDF here:


Interactive US Coast Pilot Volume Coverage


(The best digital graphic we could find adds a bit of an international touch to the program!) 


A typical volume link then shows more specifically what it covers with the chapter numbers indicated, as shown below.  Each volume page has a link to the NOAA site so you can download the volume of interest.







Wednesday, June 30, 2021

Online Global Chart Viewers

 For now, this is just a place holder where we put the links. Later we will do more study and actual comparison. For now, this is where you can go to see what the nautical chart of an area looks like... or might look like, since not all are official outlets.

These two are third party, not official, but indeed global

Global via Navionics

Global via Aqua Map

Official charts can be seen here.

US ENC charts   

US RNC charts  (they are closing this down on Oct 1, 2021)

Meso American-Caribbean Sea  

Here we see where the charts exist, but not the actual charts

Canadian charts (ENC and RNC)  (Not much details of what is actually on the charts, but shows where they are.)

Similar global coverage from IHO   (Shows where charts exist, but can't actually see them.)

_________

We have an article on Free ENC Viewers

We have a more in-depth index to US products here:  www.starpath.com/getcharts


Friday, June 18, 2021

Sharing Routes Between OpenCPN and qtVlm

OpenCPN and qtVlm are the two most popular navigation programs in the world, with thousands of international users. They are free programs, both with options to make donations to help cover expenses and support the developers. They each have both PC and Mac versions, along with other options, including mobile versions.

Each has its own unique and powerful features, many of which we don't find elsewhere, or maybe only in high-end commercial products. Because they are both sound, working tools with unique strengths, we use both in our online navigation and weather training courses at Starpath. 

Common to both, of course, is the option for user-created routes made up of a sequence of waypoints. Essentially any navigation program, not just these two, include this operation in some form, which is indeed a requirement of the International Maritime Organization (IMO) for an approved navigation program. If you are not following a route, you are not navigating; you are just out sailing. And essentially all navigation programs offer a way to export a final route in the form of a GPX file, which can in turn be imported into other navigation programs.

But OpenCPN and qtVlm have jointly gone a step beyond this in that each includes the functionality to just copy a route from the chart screen and paste it onto the chart screen of the other program. The reason this is so valuable is tied to the way each program creates the routes. Each has specific strengths you may want to take advantage of for the route creation, even though you choose to navigate that route in the other program. 

qtVlm, for example, is oriented toward sailboat navigation under sail. It offers several fast, easy ways to create a sailing route based on the vessel's polar diagram and a loaded grib file of forecasted wind—tools that do not require running a full isochronal routing computation. But those tools are not a bonus when you know you must be under power for most of the voyage.

OpenCPN, on the other hand, offers more convenient waypoint placements and adjustments, with quick chart display changes, so it offers a faster way to make a fine tuned route through a complex waterway that will be traversed under power.

Thus we may want to use one or the other programs for setting up our route, depending on the conditions, and then having the route in hand, we want to paste it into the other program.  In very broad terms, OpenCPN is likely best choice for complex routing under power and qtVlm is the best choice for figuring a route that must be followed mostly under sail.

In the common case of wanting to sail as much as possible, but knowing we have to go under power along parts of a complex route, maybe many days long, then we might want to create and fine tune the route in OpenCPN then paste it into qtVlm to see how much of it we might actually sail when we learn the actual wind forecasts when we are there.

Also, there are different protocols for following an activated route in the two programs, as well different simulation options, and different interactions with AIS targets. So even if we are set on actually using one over the other once underway, it could be we learn a lot about navigation of the route by simulating the following of the route in each of the programs.

Another example of cross program navigation, without even considering the distinctions between power and sail,  could be the choice to use OpenCPN for routine chart navigation to take advantage of its convenient interface of controls and chart display, while running qtVlm simultaneously to monitor and evaluate the weather along the route since it has so many powerful and unique weather related features. This type of operation greatly benefits from the ability to copy and paste a route from one to the other program.

So we are lucky that these two programs have collaborated to let us just right click a route, copy it, and then paste it into the other program, as if we had made it there.

And, while here, may I take the opportunity to thank the developers of both programs for the fine products they have made available to the public. This is a huge benefit to mariners worldwide. 

Later, individual mariners may choose one of the long-tested commercial products such as Expedition, Coastal Explorer, LuckGrib, and TimeZero. These products have earned their top of the line ranking. We use all of these ourselves in various applications, as do top navigators worldwide. But we cannot overlook the valuable role of the two free programs we discuss here for the introduction and training they provide mariners on the power of echart navigation.

Below is a video demonstration of the process.




Thursday, June 3, 2021

HRRR 48-hr forecasts and sub-hourly forecasts

For some time now, the HRRR model forecast has had options to its base offering  of updates every hour, with forecasts extending out to 18hr. One extension is a longer term forecast updated every 6 hrs, extending out to 48hr, but not many popular sources had the extended data available.  Now we have several sources, so this detail might benefit from a highlight, not to mention that in some cases the presentation might be confusing.

LuckGrib

LuckGrib for (Mac or iOS) pioneered the access to this data, and still has the clearest interface to the options, plus LuckGrib also offers access to the unique HRRR sub-hourly wind forecasts (every 15 minutes for 18 hr), which are not available elsewhere to my knowledge—other than a direct download from NOAA, which takes special procedures. The LuckGrib solution is to offer three separate model choices:

(1) HRRR hourly forecasts, out to 18h, updated hourly, delayed 1h 30m.

(2) HRRR 15 min forecasts, out to 18h, updated hourly, delayed 1h 28m (wind and gust only)

(3) HRRR hourly forecasts, out to 48h, updated every 6 hr at the synoptic times, delayed 2h 01m.

These are the most timely forecasts we have, but there is still a latency to account for the model run time plus the 10 or 15 min it takes LuckGrib or other third parties to process the data once ready.  

This means that if you request (1) at 1500, the first forecast you get would be valid at 13z and the last would be 18h later at 07z the next day. Noting the delays to the minute allows you to get new data about half an hour earlier, but you would still be comparing present observations to a forecast that is 1.5 hr old.

The same is true with the sub-hour forecasts (2), but at the earliest you will always get at least 6 forecasts that cover past times. Your logbook records of these winds and then a good check on the actual forecasts going forward in time.

For inland routing or planning a day sail or a race, option (3) is enticing, but the timing might be more crucial if you want the freshest forecast.  Requesting (3) at 1410, would bring 48 hourly forecasts starting at 12z today and ending at 12z in two days. You would have only lost 2 hr of currency, which is unavoidable.  On the other hand, if you requested (3) at 1330 or even 1400 exactly, the first synoptic time earlier than the 2h 01m latency would be 0600, so you would be working with a forecast that is 8 hr old. 

In short, when using the extended forecast HRRR data we must be aware of the latency. Note that the fact that this runs at only the synoptic times means it will make an interesting (timely) comparison with the  3-km NAM forecasts as well as the NBM CONUS data.

Saildocs

Saildocs does this a different way. They have only one HRRR model request, but it can be used for the 18h data or the 48h data.  The result you get back depends on the extent of the forecasts you ask for.  If you ask for less than 18h of data,  ie every hour (or every 3h) out to 18h then this is interpreted as the 18h run computed every hour with a latency of 1.5h.  If you ask for any forecast beyond 18h, then that will trigger the 48-hr forecast  run only at the synoptic times, now with a latency of 2.0 hr. 

In short, it is about the same as LuckGrib on the extended forecast, but Saildocs does not have the sub-hourly forecasts.

Expedition

Expedition uses Saildocs for access to the HRRR, and to accommodate the 48h option, it shows extended hours for choosing the number of forecasts, and hence the trigger to the type of data. It works as explained above, but users need to know the conventions and latencies.

Other Viewers and Apps

XyGrib does not access HRRR. OpenCPN, qtVlm, and others that use Saildocs show the HRRR interface but are not updated yet, and still assume the maximum hours for HRRR is 18. These programs often generate an email for the user, that we then send to query@saildocs.com to get the file, which we then import as an external grib file.  To use this same email for the 48h forecast, just change the end forecast date from 18 to 48 or 30, etc.  Keep in mind this is high res data, so the files can get huge fast. We have to make judicious choices of region and parameters or hit server limits on file size.

Some mobile apps allow the choice of HRRR, but default to the 48hr data for all selections, so users must be aware that there is maybe newer data from that model.

***

Numerical weather models including regional models for inland sailing are discussed in our textbook Modern Marine Weather.



Monday, May 3, 2021

ASCAT News — The Good and the Bad.

Scatterometer wind measurements are the truth meter for weather work at sea. There are several sources, but the ASCAT data from EUMETSAT are the gold standard. We can see this data graphically at the STAR website—found easily with a Google search of "ASCAT." We have extensive coverage of this data and how to use it in Modern Marine Weather (MMWX), but the book links are out of date (being updated here!)—in part because there are now three satellites with more data to be presented, Metop A, B, and C.

So the first news is we have the updated links below, there are new data, and it is even better than before. The main bad news is the GRIB format of this data, which offered a super convenient and precise way to test model forecasts is at present no longer available from the Ocean Prediction Center (OPC). It is still listed at the OPC website, which might imply the loss is not permanent. The files look normal and are indeed updated as indicated, but they do not contain any data!

Granted, this wonderful source of digital data could only be conveniently viewed in two popular commercial programs, namely Expedition and LuckGrib—not to mention the free app Panoply, since that one takes extra work. Worse than that, even amongst the many users of these two leading programs, there were not many navigators actually taking advantage of this most important data, which might account for the lingering loss... ie there are not many people complaining. 

(For now, there is no GRIB source for ASCAT, but for those who want it enough to spend some extra work you can download the data in netcdf and then view it in Panoply, but this is not a practical, navigator's solution.)

Part of the problem in ASCAT popularity is the data are not plug and play like a GFS forecast is, for example. Once we did get the ASCAT grib, life was good, it showed the actual winds on the ocean, not a forecast, but real data as if we are looking at anemometers on a field buoys located every 25 km across the ocean. The problem is the data over some specific point, say 300 nmi in radius, might only be updated 2 or 3 times a day, and those times take some effort to predict. Refer to MMWX for these details.

The good news is, and the motivation of this article is, we have a new, free way to get georeferenced ASCAT data into a navigation program when underway using the program qtVlm. We already had one way described in MMWX, namely look up the file name for the data you want from the STAR site or from the table in MMWX, and then request that image from Saildocs. Once that is in hand, and you check the valid time of the satellite pass (MMWX), you can either just look at it to see the winds and compare to the forecasts, or in Expedition, OpenCPN, or qtVlm  you could manually georeference it and then overlay the model forecast at the corresponding tine right on top of it. In these programs you can store the georeference coordinates and then just periodically update to see if there is new data. The image size you are requesting is 20 to 60m kb. A sample is below.


ASCAT-A descending over Bermuda at 12:33z on May 2, 2021. It is May 2, and not May 3, because this image is from the 22-hr dataset that was most recently updated at 02:07 May 2. Had the small purple time at the bottom been between 00:00z and 02:07z then this would be May 2.


These files are always 10º Lat x 15º Lon so they are easy to georeference, plus the grid lines are clearly in view.

Below are samples georeferenced in Expedition and qtVlm with the GFS wind forecast at the corresponding time overlaid on it.


And then zoomed in below, we see the GFS is pretty good since it agrees with the scatterometer wind measurements. This is Metop A, descending. 


The same thing viewed in qtVlm looks like this:





The best comparison of overlaid data like this depends on our choices of transparency and wind barb  colors. This has not been optimized in either of these two programs.

The main new good news from qtVlm is their latest version allows us to enter a live internet link for the image, as opposed to a link to a static file on our hard drive, so that the data are updated automatically when we reload that image. The trick is to create an auto-updating KML file for image and then use that as the target file. This then not only updates automatically, it also brings with it all of the georeferencing information. With your computer linked to your satellite phone, this update should take place just as you do to obtain the latest model forecast.

We have videos online on how to do this.  We make the needed files in Google Earth, which is then another way to look at live, auto updating ASCAT images. The new step is just adding this functionality to a navigation program independent of Google Earth.

We made such KML files  some years ago for the Bermuda Race,  Transpac/Pacific Cup, and for Sydney-Hobart. Our next step will be to update the links and make a convenient place for mariners to download them.  Now that we have lost the GRIB format, these methods become of interest again.

Here is the generic image link using the Bermuda file as the base. Then the colored letters have to be changed for other locations, passes, and satellites.

https://manati.star.nesdis.noaa.gov/ascat_images/cur_25km_META/zooms/WMBds86.png

Change A to B or C for the 3 ASCAT satellites, and change the blue file name to match the location you care about, and then change the d (descending) to a for the ascending pass. 

Thus for each location you care about, there will be 6 files that might be of interest, keeping in mind that A and B are fairly close in time so the earth has not rotated much between them. In rare cases, you might need B having seen A, but not often.  

You can get these files (once you know the name of the ones you want) from Saildocs. For example, send an otherwise blank email to query@saildocs with this as the message:

send https://manati.star.nesdis.noaa.gov/ascat_images/cur_25km_META/zooms/WMBds86.png
send https://manati.star.nesdis.noaa.gov/ascat_images/cur_25km_META/zooms/WMBas86.png

This should get you those two files by return email. Sample file names below.


Click the images above for a better view.

Below are a couple How to videos.



How to create KML files of ASCAT wind data.



Viewing ASCAT wind data in qtVlm

Wednesday, March 17, 2021

Timing and Time Structure in Model Forecast Gribs

Analysis and forecast timing is crucial for good weather work. This is not just a challenge with model forecasts in grib format (the subject at hand), we also face this when using regular graphic weather maps and text forecasts. We cover this in Modern Marine Weather, under the heading "weather map sequencing" near Table 7.2-1. 

The timing involved for the GFS and other global models is very roughly outlined below. (Regional and rapid refresh models have a totally different timing pattern.)

(1) At precisely the synoptic times (HH = 00z, 06z, 12z, and 18z), observers and instruments around the world collect observations and report them via a global network to all the labs around the world running numerical weather prediction models. (A time label "z" means zulu time, which is a nickname for UTC, which was once called GMT.)

(2) The models collect the data and analyze it for quality and then assimilate it into their model programs and start a new global computation. We are now at HH + 2 hr or so

(3) The model completes the global surface analysis and passes it on to other labs. This is now roughly HH + 2.5 hr. Now, or a bit earlier, the model starts computing the various forecasts based on this analysis.

(4) The OPC receives this latest model analysis, which they combine with their own analysis and create the official OPC surface analysis graphic maps and text reports. This is now roughly HH +3.5 hr

(5) The model completes its full extent of forecasts and posts these online for third party access. This is now about HH + 5 hr 10 min. 

(6) Third party providers (Saildocs, Expedition, LuckGrib, XyGrib, etc) complete their download of the analysis and forecasts to their own servers and process it as needed so they can distribute this to their users. This takes us to about HH + 5 to 6 hr depending on several factors, discussed below.

Data are available to us following steps (4) and (6), with actual times depending on the format, product, and source we use, and we have to, of course, keep in mind both local times and dates, as well as the UTC times and dates of the products.  

In the following we first look at the time structure of what we get, and then address the question of when we can actually get it.


What Forecasts Are Available?

When we ask for a set of model forecasts, it is generally assumed that you want the most recent data. Some data sources let us choose between forecasts based on the latest, or forecasts based on a specific synoptic time over the past 24 hr, which may not be the latest. This latter option would only be used in special circumstances, such as comparing with past ASCAT wind data, or for filling in something we missed in the past 24 hr. Those offering this past data generally only go back 24 hr. (If we want even older grib data we have to look into archived or reanalyzed data, which is a topic of its own.)

When we ask for model forecasts such as GFS, we specify the Lat-Lon region, the extent of the forecasts, i.e., out 1 day, 2 day... on up to 16 day for GFS, and we specify the forecast interval, every hour, 3 hr, 6 hr, 12 hr etc. If we are asking for data from Saildocs by email, we must know ahead of time the extent and interval sizes that are available. When using data sources from within a grib viewer, they usually have the intervals and extents listed and we choose from what is available.

What is often not so clear, even when using a grib viewer with selection options is the available intervals typically change with the extent of the forecast, and the options we have depend on the source we are using. If we use the GFS model as an example,  NOAA creates these forecasts every hr for 120 hr (5 days), and then every 3 hr out to 16 days (384 hr), but outside of direct request from NOAA, there is no third-party source for data  over this full frequency range. Each third party provider makes a choice of what they believe best meets their user's needs. Samples are shown in Figure 1.

Figure 1. Sample availability of GFS forecasts.

LuckGrib and Expedition (via its direct NOAA link) are the only sources outside of NOAA direct that offer the 1hr data. XyGrib and those who use it only go out 10 days, but they offer 3 hr steps over that full range. Ten days is certainly more than adequate, except maybe for an initial long look ahead for planning. Generally GFS forecasts are good out 4 days, but then fall off. For longer term forecasts, the Oceanic National Blend of Models is likely best.

When planning a departure time, data every 12 hr could be adequate; when doing an optimum routing, you might want data as frequently as available within your airtime budget—but there are other factors to keep in perspective when making that choice.

The question of when we can actually download or request these various forecasts is another topic, covered below.


Time Format in Grib Forecasts

When we ask our grib viewer or other source for the latest forecast extending out, say, 48 hr with forecasts, say, every 6 hr, for wind, pressure, and rain, we can expect that each of the forecasts we get back will include the same parameters, except for the rain. A generic "rain" request is likely to be given in terms of accumulation, and there will not be any rain data in the first forecast, as none has accumulated at that point. (GRIB is a WMO weather based computer format, and rain is a deceptively complex parameter in that system. See notes in Modern Marine Weather.)

This 48 hr forecast in 6 hr steps will show up as one file containing 9 weather maps. The first is a GFS surface analysis, valid at the synoptic time of the model run, followed (in this example) by 8 GFS forecasts, one for every 6 hr past the synoptic time of the run, out to 48 hr. If the synoptic time of the run was 12z on Mar 13th, the last of the maps would be for 12z on Mar 15th.

These files might be identified in your grib viewer or nav program two ways. All programs will list them by their valid times, such as 18z Mar 14 or 00z Mar 15, but you may also see them identified by the extent of the forecast. In that presentation, the surface analysis is called h0, the 6-hr forecast is called h6, the 12-hr forecast h12, and so on. 

Figure 2. Time structure in a set of grib model forecasts.

In this example, based on a 12z run on Mar 13th, the 18z forecast on Mar 14 would be h30; 00z Mar 15 would be h36. In practice we really need to know both identities, when it is valid and how old is the forecast. Some programs (i.e., Expedition) use h6, h12, etc; others (i.e., LuckGrib) use 6h, 12h, etc for this notation.

Each grib or nav app will have its own unique way to step through these forecasts. There will be a way to go from one forecast to the next, or go to any specific specific forecast. Also, since these are digital data, the programs can interpolate the times. You may only have forecasts for 12z and 18z but you can ask for an interpolated map at say 14z, or even 1437z, which you might want, for example, to compare with the wind measurements from an ASCAT satellite that went by at that time. Or you could want to compare with data in your logbook, or data from a buoy given at some specific time. The programs do not specifically warn you that these intermediate times are interpolated, but they are, just as the wind data itself is interpolated at points between the actual grid points for the forecast resolution.


When Can We Get the Latest Model Forecasts?

This timing is important for a couple reasons. First we want to be sure we are comparing the right forecast with the right observations, and second, we do not want to download a new file and spend the satphone airtime just to get back the same forecasts we got a couple hours ago. Also when racing, you might want to make a decision as soon as possible, which means getting the newest forecast as soon as possible.

The fundamental fact we must live with is, the earliest possible weather analysis we can get is going to be some hours old, so to compare our observations with that analysis means we have to rely on our logbook records that go back to that time... or use saved tables or histograms of our instrument readings. The analysis will always apply to a synoptic time, so it is fundamental to record, one way or another, all weather data at each of the synoptic times. Wind speed, wind direction, and the barometer (and the trends of each) are the main data, but estimated sea state, measured ocean current, state of the rain, and sea surface temp are other parameters that can matter, as well as cloud cover. Air temp might be useful in the midlatitudes to investigate frontal passage.

To compare what we are seeing on the water at the moment, we have to look at an interpolated forecast that applies to the present time, as noted above.

If we assume we are not relying on HF rfax weather map transmissions, which are only available at specific times of day (a huge drawback!), then we are considering now the earliest times we can request the data (with the understanding that we could get it any time after that).

When OPC issues their analysis (Step 4 above), we know that they have the model data in hand and have had time to put all the data together. We can learn exact values of how long it takes them to issue the latest graphic analysis maps from the annotated rfax schedule at OPC, a sample of which is below.

Figure 3. Sample of an rfax schedule showing broadcast (left) and available times (right).

Part 2 (West North Atlantic along the US coast) was broadcast at 2138z on rfax (left-most time), but it was actually available to download direct from NWS via FTPmail or Saildocs at 2112z (right-most time) a bit earlier. Thus this graphic map analysis is 2112 - 1800 = 3 hr 12 min delayed, leading to the typical 3.5 hr delay for these graphic products—it will take about 15 min to request it and get it back if all goes well.

These 18z analysis and forecasts from the GFS model in grib format, however, will not be available on the boat or at home for another couple hours. These 18z GFS forecasts from LuckGrib were available at 2320z and available from XyGrib a bit earlier at 2317z. These were about 5h and 20m delayed, which is typical for global forecasts. You can see the most recent delay times for any model via LuckGrib at luckgrib.com/status. XyGrib delay times for the models they provide are available from a menu link within the program. LuckGrib will warn you if you attempt to download data that are not yet updated, providing the second request is identical to the first one.

A working estimate for data delays of OPC graphic analysis is about 3 to 3.5 hr after the synoptic time, and we get the first grib versions of global models such as GFS in about 5 to 6 hr after the synoptic times.

There are, however, a couple nuances to this latter delay that you might run into if you dive further into the process. If we look into the actual computation times of the various models, which we can see at this NCEP link, we note two things. First the GFS analysis computations are typically done at about HH+3h 22m, but the full range of forecasts out 16 days (384 hr) finishes at about HH + 5h 10m. The  intermediate forecasts are completed at intermediate times.

Thus our grib providers could actually provide the GFS analysis (h0) at about HH + 3.5 hr, but the full range of associated forecast would not be ready for another 3 hr or so. If you ask for a 6-day forecast at, say, HH+4 you would get the latest analysis and maybe a couple of the latest forecasts, but the later forecasts you got would be from the previous model run from HH-6, because they had not been completed by HH+4. 

To avoid that potential confusion LuckGrib, and likely other data providers, do not take the data as it completes, but wait the full 5h 10m to start their downloads and processing. This way users get all the latest forecasts referenced to the same model run at the cost of not getting any at the very earliest time possible. In the case of LuckGrib, the global model delays are about 5 to 6 hr.

Another way to learn the actual times the files are available is to look directly at the NOMADS link where they come from. The times listed are the times these files were posted. These selected forecasts, for example, were based on the 12z synoptic run (.t12z) with 0.25º resolution (0p25):

gfs.t12z.pgrb2.0p25.f000                  16-Mar-2021 15:27  289M  
gfs.t12z.pgrb2.0p25.f120                  16-Mar-2021 16:03  331M  
gfs.t12z.pgrb2.0p25.f240                  16-Mar-2021 16:33  330M  
gfs.t12z.pgrb2.0p25.f384                  16-Mar-2021 17:09  325M  

The 12z analysis (f000) was posted at 1527z; the 5-day (f120) was posted at 1603z; the 10-day (f240) was posted at 1633z, and the longest forecast at 16 days (f384) was done and posted at 1709z. This represents the average of 5h 10m for the full set. We anticipate a new GFS v16 in the near future, which might expand this delay another 10 minutes or so.

Grib Data Distribution is Not Guaranteed

NOAA uses two broad categories to describe the development and distribution of data: Operational and Experimental. The model forecasts we are using are all Operational; indeed a primary source of the data is the NOMADS site, which stands for NOAA Operational Model Archive and Distribution System. So the data are certainly operational, but the distribution (dissemination) of the data is still not guaranteed, as we read in this note at the NOMADS site. We see similar disclaimers at almost all NWS sites. The only "official NWS dissemination systems" are NOAA Weather Radio and data obtained directly from the local forecast offices. In short, the main data we care about is not guaranteed to be available to us, and that does not even account for the third-party processing that we also rely on. 

This means we have to be aware that the model forecasts in grib format and other grib products from  around the world may in rare cases not be complete or may be missing. Speaking with experts who know the details, I learned that NOMADS has been remarkably dependable over the years. A recent, rare snag in the system was addressed promptly and users were kept appraised of the situation at all times.

To me, a way more disconcerting example is the recent, unannounced loss of the ASCAT data in grib format. This has still not been announced, nor a remedy proposed.

Another fringe example is the grib version of the NDFD, which are the official NWS forecasts converted to digital format. These latest data include inputs from numerous local offices and as a result the format and timing in some user selected regions (that might span two offices) are occasionally inconsistent. This is not a global product, so we will cover it when we discuss regional grib files.

The main message is, although the data always look very official and consistent, we might periodically see glitches in the products, such as missing parameters, or fewer forecasts than anticipated. This is all based on computer technology, which is very good, but none is 100% dependable... if we want 100% we should go to NOAA Weather Radio, but we better use a radio that is 100% guaranteed to work right.