Monday, July 8, 2019

FTPmail No Longer Includes Observations—Plus Alternatives

For decades, mariners had a convenient way to request near-live wind, pressure, and sea state observations from buoys, as well as data from land based stations. This was achieved via the NWS FTPmail service, accessing data in the folder


Using standard FTPmail protocol, we could then send an email request to ask for a file such as 51001.txt (a HI buoy), wpow1.txt (West Point Lighthouse, near us), or any other buoy or station operated by NDBC.  This would provide a short text report of the latest data from that station. These were updated every 10 minutes.

The process was explained along with a list of the stations in a file called buoydata.txt. The file is still there (at least for now), but the first part that explains the process is no longer valid.  The list of stations, remains useful, but the notes on "present conditions" is long out of date.

We learned today, however, that the NDBC is no longer updating the data files in the FTPmail folder. It seems they stopped doing this, more or less without notice, at the end of 2018.  This source was still listed at the Ocean Prediction Center until today, when they removed the information at our request because this source is permanently gone.

We are trying to learn why NDBC discontinued transfering the data to the FTPmail folder, ending access via FTPmail, and now post it only on their own server, without any direct reference to it.

Going forward... 

We can still get these data by email request, but we have to go directly to the NDBC for it, and we have to rely on the good help of saildocs.com, a non-governmental organization of sailors.

We have several options, all of which are tied to the new folder the NDBC is using to store the files, namely:

One method is similar to what we historically got from FTPmail, namely latest report for just one station. These would be the same file names we used in FTPmail, namely, for example, wpow1.txt or 51001.txt, which include the following.
Station WPOW1
47° 39.7' N 122° 26.1' W 5:00 pm PDT 07/10/19
0000 GMT 07/11/19
Wind: S (170°), 14 kt
Gust: 15 kt
Pres: 30.05 steady
Air Temp: 65.1 °F
Dew Point: 61.2 °F 
and 
Station 51001
24° 27.2' N 162° 0.0' W 2:20 pm HAST 07/10/19
0020 GMT 07/11/19
Wind: E (80°), 9.7 kt
Gust: 13.6 kt
Pres: 30.07
Air Temp: 81.0 °F
Water Temp: 82.0 °F
Dew Point: 76.8 °F 
Wave Summary
1:40 pm HAST
2300 GMT 07/10/19
Swell: 5.6 ft
Period: 12.1 sec
Direction: NNW
Wind Wave: 1.3 ft
Period: 3.7 sec
Direction: ESE

This is the same as what we got before, but now there is no longer any government email source for observations—a giant step backwards in that observations are crucial to planning and email is the most common source of weather data offshore for a vast number of vessels. To get these observations by email we need to send a mail to query@saildocs.com with the following message

send https://www.ndbc.noaa.gov/data/latest_obs/wpow1.txt
send https://www.ndbc.noaa.gov/data/latest_obs/51001.txt

Then we get back two emails with the above text reports. You can ask for just one or multiple ones. It seems that the update times are not consistent from one station to another. The wpow1 lighthouse updates only every hour at this link, whereas the 51001 buoy in HI updates every 10 minutes. 

You can also put this link in your phone


and you have access to something very similar, if you have cell phone connections.

Or...  just use Dial-a-Buoy on your phone and find the station by Lat-Lon: call 888-701-8992‬ and follow instructions. Every navigator should have that phone number in their address book!


Alternative Formats of Observations data

Most recent observations from all stations updated every 5 minutes are available at this link


A sample of this file is below


This file is about 95 kB; it gives only the latest report, but lists all NDBC stations.

We can also get all data from a specific station updated every 10 min at this link


where we attach the station name at the end, such as 5100.txt. A sample is shown below. Some stations only have data every hour; others every 10 min with wave data maybe every hour.


The data go back 45 days.  File names are in caps (if letters are included) for this folder, but lower case in the other folder, and that matters for this download or email request.  A typical file size is 104 kB, which is clearly larger than we want by satphone.

Alternatively, we can ask for  a continuous-wind file from the same folder, such as

https://www.ndbc.noaa.gov/data/realtime2/WPOW1.cwind

#YY  MM DD hh mm WDIR WSPD GDR GST GTIME
#yr  mo dy hr mn degT m/s degT m/s hhmm
2019 07 08 22 00 160  2.6 150  3.6 2104
2019 07 08 21 50 160  2.6 999 99.0 9999
2019 07 08 21 40 157  2.1 999 99.0 9999
2019 07 08 21 30 158  2.6 999 99.0 9999
2019 07 08 21 20 164  2.6 999 99.0 9999
2019 07 08 21 10 157  3.1 999 99.0 9999
2019 07 08 21 00 156  3.1 160  3.6 2047
2019 07 08 20 50 162  3.1 999 99.0 9999
2019 07 08 20 40 165  3.1 999 99.0 9999
2019 07 08 20 30 158  2.6 999 99.0 9999
2019 07 08 20 20 166  2.6 999 99.0 9999
2019 07 08 20 10 171  2.6 999 99.0 9999
2019 07 08 20 00 165  2.6 170  3.1 1958
2019 07 08 19 50 167  2.6 999 99.0 9999
2019 07 08 19 40 163  2.6 999 99.0 9999
2019 07 08 19 30 157  2.6 999 99.0 9999
2019 07 08 19 20 158  2.1 999 99.0 9999
2019 07 08 19 10 161  2.1 999 99.0 9999
2019 07 08 19 00 163  1.5 160  2.1 1859
2019 07 08 18 50 178  0.5 999 99.0 9999
2019 07 08 18 40 268  0.0 999 99.0 9999
2019 07 08 18 30 308  0.5 999 99.0 9999
2019 07 08 18 20 000  0.0 999 99.0 9999
2019 07 08 18 10 319  0.5 999 99.0 9999
2019 07 08 18 00 326  0.5  40  1.5 1705
2019 07 08 17 50 328  0.5 999 99.0 9999

Here we get only wind data from only this station. This file lists the wind every 10 minutes at the station, updated every 5 min.  The file size is about 260 kB.  This is a text file, but without the standard .txt extension.  Once downloaded, highlight the file, and say open with Notepad (OC) or TextEdit (Mac).

Remember we can get any of these reports by email request to saildocs, i.e., send a mail to query@saildocs.com with one or all of these messages

send https://www.ndbc.noaa.gov/data/latest_obs/wpow1.txt
send https://www.ndbc.noaa.gov/data/latest_obs/latest_obs.txt
send https://www.ndbc.noaa.gov/data/realtime2/WPOW1.cwind

______________

The parameters for all of these files are explained at this link https://www.ndbc.noaa.gov/measdes.shtml

You can find  the best stations for your application most easily with their graphic interface at the the home page: https://www.ndbc.noaa.gov/   then make a list of the ones you care about.

Alternatively, If you do not know the name of a station, you can find available ones based on Lat and Lon at https://www.ndbc.noaa.gov/radial_search.php?

A list of stations and what products they have is here: https://www.ndbc.noaa.gov/data_availability/data_avail.php

Don't forget our powerful free email service to get live ship reports.  See starpath.com/shipreports. This provides all reports within 300 nmi of a specified Lat, Lon over the past 6 hr.

The navigation program Expedition has a unique feature of downloading and displaying all the latest buoy and ship reports, but they include only the latest data. Seeing the sequence of these is often instructive. The GPX file of reports we create in Starpath Ship Reports can be imported to any nav program.





4 comments:

Chuck B said...

This is working as of 4 Nov 2021 to retrieve NDBC observations via FTPMail:

To: NWS.FTPMail.OPS@noaa.gov

open data.ndbc.noaa.gov
cd data
cd latest_obs
get 46012.txt
quit

David Burch said...

Thanks very much. That does indeed seem to work, but it is a mystery. I will try to track this down. The FTPmail data folder online is still advertised as https://tgftp.nws.noaa.gov/data/ and that folder does not have a latest_obs folder in it, so something has changed. May I ask where or how you found this link.

David Burch said...

This means we can now get anything that is in that folder, ie
open data.ndbc.noaa.gov
cd data
cd realtime2
get 46012.txt
quit

to get back 45 days of every 10 min reports (comes in 3 or 4 emails). we would have to see what all is in here to see what else might be useful: https://www.ndbc.noaa.gov/data/

one thing i see is a list of all the reporting stations with details on their instruments.

Chuck B said...

Hi David, I had to piece it together. The main clue was this page which indicates that NDBC has an FTP server: https://www.ndbc.noaa.gov/rt_data_access.shtml

Note the page says, "this access method is expected to be discontinued," but no time frame is given.

It appears to mirror what's seen at https://www.ndbc.noaa.gov/data/.