Thursday, January 21, 2016

How to Load Weather Maps into OpenCPN — The Hard Way!

Since posting this note (and spending much time on figuring out how to do this!) we have learned from a comment that there is indeed a very much easier way. There will shortly be here a link to a new article.  The "new" method is a remarkable plugin to OpenCPN called Weatherfax, which has apparently been available for some time and I just did not know about it.  It not only allows image display with convenient georeferencing, but also includes a direct link to many maps online that show up automatically georeferenced in OpenCPN.  

So best bet is go the new article unless you have specific interest in the use of georeferenced images in the kap format as used with NOAA echarts.

-----------------------

With the latest surface analysis and forecast maps in an echart viewer, we can see our GPS position right on the map and plan routes across the weather patterns. It also facilitates reading wind and pressure from the map at places where none are shown. Another important application is overlaying the GFS grib forecasts on actual NWS forecasts to evaluate the GFS data—in areas where we do not have digitized NWS forecasts in the form of NDFD gribs, in which case this could be done more easily by just comparing gribs.

We discuss this in Modern Marine Weather in Section 7.2 called Georeferencing Weather Maps. In that discussion we had recommended the program Memory-Map Navigator as a way to do this as it was possible with their free, demo version. That program still has this functionality, but the demo version no longer includes this option.  Other commercial products also offer this, including Ocens MetMapper, and Polar View. In principle the free program Sea Clear can do this, but I have not yet succeeded with that. All of these solutions can create a weather map echart, but they would each be unique to their own viewers. The procedure below makes a file that will open in any echart viewer.

Our electronic charting and weather training now use OpenCPN for the in class exercises, so the goal at hand was to develop a method for loading weather maps into OpenCPN. The final product should in principle open in any echart viewer. The explanation of the process is outlined below, followed by a video showing each step in action.

We will use a free utility called imgkap.exe for the PC. It is used to convert images to the .kap file format that is used in the RNC echart format, also called BSB.  Once the file is made, it will open in Mac or PC versions of OpenCPN. I do not know of a Mac equivalent, but there could be something.

Below is the step by step procedure, and here is a video that shows the steps in action using a later 24h surface analysis.



Step 1. Download a copy of imgkap.exe (Windows binary, v 1.11, 3.5 MB) from dacust.com/inlandwaters/imgkap. This program is discussed at that link as well as in the OpenCPN Manual at this link opencpn.org/ocpn/imgkap.

Step 2. Make a folder on your desktop called imgkap and put imgkap.exe in it. Needless to say, you can call things what you want and store them where you want; we just need one place to put all the parts. We will need three parts: this program, the map image itself, and a header file that tells the program how to make the kap. With the nice tools made available to us, the work reduces to making the header file.... and I hope to reduce that part of the work by providing a header file for each of the standard wx maps we get from NOAA.

Step 3. Download the weather map image and save it to the imgkap folder. We use here the example of a 24h surface forecast for the Pacific, which you can get from www.opc.ncep.noaa.gov/shtml/P_24hrsfc.gif.  This gif image should be 907 x 1200 pixels. The main index to all of their maps is at www.opc.ncep.noaa.gov — click the ocean tabs to get to the maps. Below is the top part of the map we are working with.


Step 4. Convert that gif to a png using a graphics program. This can be done in any version of Windows by right clicking it, choose Open With, select Paint, then after it opens Save As, choose png, save it, and close Paint. Then you will have two images in the folder a gif and an png. You no longer need the gif and could delete it.

Step 5. Now create the header file: Open Notepad and copy this text into it:

! 24h pacific surface forecast from OPC.
VER/2.0
BSB/NA=24h-forecast
    NU=24h-forecast,RA=907,1200,DU=300
KNP/SC=53569020,GD=WGS84,PR=MERCATOR,PP=40
    PI=UNKNOWN,SP=UNKNOWN,SK=0.0,TA=90.0
    UN=NONE,SD=NONE
    DX=4535.5,DY=4535.5
OST/1
REF/1, 1,1200, 15.7,-157.0
REF/2, 1,1,62.5,-157.0
REF/3, 907,1,62.5,-108.0
REF/4, 907,1200,15.7,-108.0
CPH/0.0
PLY/1,15.7,-157.0
PLY/2,62.5,-157.0
PLY/3,62.5,-108.0
PLY/4,15.7,-108.0
DTM/0,0

Note that the indents matter; they signify that the line above it was more than 80 characters, which is the limit. No blank lines before this text and none after it.

Then save this notepad file in the imgkap folder and name it P_24hrsfc_header.kap.

Step 6. Now we are ready to make the kap file. Now open a cmd window, ie from the Start button or equivalent type cmd and enter and the window will popup.  Then type cd desktop/imgkap and enter and you should see something like this:




Then type   imgkap P_24hrsfc.png P_24hrsfc_header.kap filename.kap   and then enter. Filename is anything without spaces you want to name the map, ie 24h_valid_12z_jan16.kap.  You can copy and paste this into the cmd window if you like with right click and paste—assuming you are using the same file names. Put another way, you can make your own generic file name convention and just use that each time as long as you have the right extensions and list them in the right order. You will get a clear error msg saying what is wrong if you make an error.

Or you can just type imgkap and enter to see the inputs and the other things you can to with this utility, ie you get a help file.

Then if you do not get error messages you will have another file in the folder called filename.kap.  This is the file that will open in OpenCPN as a georeferenced wx map. Here is what you see after a successful map.



At this point, we need to just move the newly created file into a place the echart program can read it as shown in the video.

Here is what the map looks like in OpenCPN.


A final note on use of this type of home made kap file in other programs. Some echart viewers require a second descriptive file be created as well called (in analogy to above) filename.bsb.  OpenCPN does not require this; it just flashes a note that there is no information on the chart. The .bsb files are relatively easy to construct from the information in the header file. I will leave that till it comes up in a specific need.

For example, one of our favorite echart programs is Coastal Explorer from Rose Point Navigation. This program does require the bsb file.  These bsb files are most important for programs that are monitoring your charts to be sure they are up to date. They are also used in ECS functionality that links hotspots on the chart to announcements or corresponding Coast Pilot data and so on.

There is a note on how to make the header files here, but once you have one you can use it without knowing the details of how they are made.


§§§


11 comments:

Captain Jack said...

This is ridiculous pain in the ass. Why can't someone just build a converter inside of OpenCPN to read almost any image? Just like this company does: http://www.macgpspro.com/?id=143

David Burch said...

I did not know about that product but will have a look. Thanks for the information. I am sure many of our readers are glad to see another fine program for the Mac.

It does not, however, look like it will do the job we are after. We need to get the weather maps into an echart program that will read both regular noaa charts as well as overlay grib formatted model forecasts. It does not look like that product will do either, but it will clearly georeference a weather map if that is all that is needed. It does say that it will load pdfs, which makes it very interesting because NOAA has their charts in pdf format, and this could be the perfect tool for that application. I will have to see if i can get a copy of that to test that point. thanks again.

There are indeed several programs that will do a nice job of georeferencing a weather map. The program called Memory-Map Navigator (memory-map.com), which is an excellent echart program, will georeference any maps including weather maps, but it is $100 and it does not support grib overlays. It looks like the one you reference is about $60, which could make it an interesting option for navigation if it will read noaa chart pdfs…. then it just has to add tides and currents and grib files and it could be a contender for marine applications.

But with that said… i agree completely with your description of the process! and we would also like to see some plugin to OpenCPN.

We are however getting this a bit better dialed in. See our latest note at http://davidburchnavigation.blogspot.com/2016/01/converting-images-to-bsb-echarts-with.html which off hand looks *more* complex, but in fact is very fast and easy to do once the 3 utility programs are in one folder.

Note that if the investment is not a prohibiting factor, then a clear choice for this solution is Expedition from expeditionmarine.com This program will import and georeference a weather map in well less than a minute, and then you have it to use for any navigation application within a world class echart program.

Very shortly I will post a note with video on the importance of this process (i.e. importing weather maps to echart programs) and give examples of their use.

We are also looking into building a java app that lets you just browse for a standard NOAA weather map on your computer and in one step export it as a BSB chart. if there are programmers reading this who might be interested we are glad to pass on the idea and details for someone else to do it.

David Burch said...

I want to stress something that may not be clear. The above article and the one linked in it about the header file creation is just to show how the these headers are made. Our goal is to actually make and publish a set of headers, one for each of the standard NOAA weather maps. Once that is done, then the process of making a chart from a map will always be the same, and it will be accomplished from one command using imgkap... with or without a neat java interface. You never have to look at the map at all, and never have to worry about these header files.

Unknown said...

use the weather fax plugin

Unknown said...

use the weather fax plugin

it already solved all of this years ago

David Burch said...

Thanks very much. I am very embarrassed to say I did not know about that. I will check this immediately today. We consider this an important operation; this function alone would justify installing opencpn.

David Burch said...

OK. I have checked this and it is amazing! You will notice that I changed the name of the article and added a note in the introduction about a new easy method to follow. The features of this plugin are remarkable; every mariner doing weather analysis should have this. I will look into the history of the plugin. The one online is dated about 4 months ago, but that is likely an update to an earlier version. And again, thanks very much for your comment here, and frankly for your politeness in the way you put it! I should have known about this. We learn more everyday about how much opencpn has evolved over the years since we had first looked at it.

Peter said...

You announced on the top a new article about the WeatherFax plugin for open cpm and how to use it.
Do you have any idea when the article will be posted?
Great fan of your weather book!

David Burch said...

Thanks for the reminder and kind words about the book. I have added this to the list. We also have on the horizon several notes and videos on the ENC display of OpenCPN. This entire article above was an effort to do something that has a much easier solution, that i simply did not know at the time.

rgleason said...

OpenCPN Weatherfax_pi has been around a very long time now.

David Burch said...

I agree completely. We actually have online for a long time now several videos on how to use the wefax plugin to load maps and other images into opencpn. These new links should have been added here a long time ago. There are also two other related articles that should go as well. Thanks for the reminder. I will address this today.