Analysis of a Celestial Navigation Sight Session

This note is background for another article to be posted shortly by Richard Rice and me entitled "Most Likely Position (MLP) from Three LOPs."  We use the sights below as one example in that article, but do not include these details that must precede an optimized cel nav fix. We go over this process here and present a free Windows app for computing MLP along with a work form for solving for it manually.

We are analyzing a sight session from the book Hawaii by Sextant (HBS), which is a record of the last voyage I did by pure cel nav.  It was in July 1982, Victoria, BC to Maui, HI. No electronics at all except RDF, and that did not work well. The lack of electronic navigation was not a choice willfully made; there simply were no options at the time, which in light of how far we have come, was not that long ago. There are many sight sessions in the book; all are analyzed in detail. It is set up as an exercise in ocean navigation that is intended to be used as a training tool to master skills in cel nav and ocean navigation in general.  When you have nothing but cel nav to go by, it is important to do the best you can with each sight, and to maintain good logbook procedures.  This of course remains true today.

Here is a picture of how that fix might show up on a plotting sheet of the DR track. We are looking into Jupiter-Vega-Altair fix at Log 2614. And indeed—in light of the present topic—we do not have this fix plotted in the best possible position. In the actual voyage we just took a center value of the intersecting LOPs as the fix. Most star sights were pretty good, so this was not an issue at the time, but now we want to concentrate on doing best possible analysis of a single sight session. This process can be helpful when sights are limited or in question, and it is the only fair way to evaluate what we can and cannot achieve with cel nav.  This type of extra work is not often required in mid ocean, but could be valuable on approaches.

Figure 1

(In the book HBS, the fixes along the DR track as shown above were not made from the optimum choice of sights. Each body, as seen below, had several sights, and it is the task of the navigator to analyze these to come up with the best representative of the full set. That process is left as an exercise in the book, with detailed instructions and many examples. The sights used for the sample plots (such as the one above) were selected almost randomly from the sights taken, with the hopes that the readers strive to improve the choices and overall navigation. In short, just one sight of each body was selected, so it had no benefit at all of the others taken. The plots included are just intended to show what the layout of the DR track would look like—although the plotting is accurate, once the choices were made.)

[ Note in passing, since it shows on the plot sheet: The narrow running fix at 1227 was clearly not a very good one as plotted here, but it got corrected with a long LAN measurement with plenty of sights on both sides of LAN to get a good fix. Had we just carried on with good DR we would have been at the 2240 location fairly well. On the other hand, that running fix was not as bad as shown in this plot if it had been analyzed more carefully. The plots in the book, again, just take two random sights from each session to show a full work form and plot. We encourage readers to improve on what is shown, given the full set of data. ]

The actual 1982 navigation logbook page of the log 2614 sight session showing analysis done at the time is below.

Figure 2

First some background notes.  The actual sight reduction at the time was done with an early version of the HP-41 nav calculator that we added a few of our own functions to. With this type of sight reduction we do not use an assumed position, but do all reductions from a common DR position, given at the top of the page.

The vessel was moving at 7.3 kts on course 227 T. The average speed during the sight session, which lasted 41 minutes, was figured by subtracting log readings taken before and after the session. A standard procedure when preparing to advance all sights to a common time.

The altitude intercepts (a-values) in black pencil were done without advancing the individual lines to a common time.  The red pencil are the advanced values, which were figured from a = a + D*Cos(C-Zn), which is a mathematical way to advance the sights explained in the book. They are all advanced to the time of the last sight at 2240 watch time.

It appears that the last Vega sight was discounted at the time because it was so far off the others. In retrospect, that was a mistake!  If you advance that one you get a = 3.6 A, which is about the same as one that I did keep at the time.  (Below we see describe the argument used in HBS for throwing both of them out, but that was not applied to this session when underway.)

For the first step in this analysis now we use all of the sights.  It is crucial that they be advanced. The fix in this case will be wrong by a couple miles if that is not done.  It is taken for granted that all sights must be advanced to a common time.

When the sights are advanced to a common time, we can (as a first approximation) just average them to get an average LOP for each body... assuming the Zn has not changed more than a degree, which is the case here.

So with that averaging of the red ones we get:

Jupiter: a = 3.0' A 200
Vega: a = 3.0' A 058
Altair: a = 5.1' A 090

Note that including the last Vega sight we threw out underway changed the a-value from 2.8 (in the logbook) to 3.0.

These sights are shown below as the light purple lines. The blue ones actually plotted were from our fit-slope analysis, described below. This is an improvement over simple averaging.

Figure 3

Some  background on this plotting:  When doing sight reduction by calculator from a common DR position, the plotting is very easy. We just make a plotting sheet centered on the DR, expanded as needed. In this case, what is normally taken as 60 nmi between parallels, we just change to 6 nmi between parallels. This plotting sheet section shown is about 12 nmi square. Then the plotting is very fast an accurate, all done from the center of the compass rose, with a scale marked in tenths of a mile.

In HBS, we explain what we call the fit-slope method. That is a way to decide from a set of sights which ones are the most consistent with the known slope of the star height versus time. In other words, for five sights taken from five different DR positions (the boat is moving), we can calculate what the heights should be if were exactly at those positions. We don't expect to get exactly those values as we do not know if we were on that track or not, but the slope of these computed sights will be same for anywhere near those locations. To find the slope, we don't need to calculate all of them; we just need a calculation of height from a convenient time before and after the sight session.

When we apply that analysis we get more insight into which sights were likely good, and more to the point, which ones were likely not as good. If we have 3 sights that increased at the right slope and one that was notably off of it, then we can throw that one out and average the rest. Or sometimes we get one that if off and throw it out, and then the others scatter above an below a line at the right slope, so we can average those, or better still, take one that is right on the line.  Doing this we get the a-values below, which are plotted as the blue triangle above. We get a smaller triangle at a slightly different location. That alone is not justification for the method, which has a sounder analytical basis. We encourage anyone who wants to improve their cel nav accuracy to look into the fit-slope method, explained in detail with many examples in HBS.

Jupiter: a = 2.7' A 200
Vega: a = 2.6' A 058
Altair: a = 4.7' A 090

The fit-slope analysis almost always improves the sights. It is effectively a more logical way to do the averaging of the sights. It also demonstrates why it is so crucial to take 4 or 5 sights of each star. It is far better to take 4 or 5 sights of three well positioned stars that it is to take 1 or 2 sights of 10 different stars.

Below, for example, shows all the sights plotted (the red a-values in the logbook), with the fit-slope choices of LOPs now marked in light blue.

Figure 4.

It seems one could do some sort of filtering on this display alone, but the results of the fit-slope method do not always match what we might conclude from such a plot of all sights.   In short, you cannot tell by the spread of the sights alone, which are the good ones. However, we will indeed use the spread (standard deviation) of these multiple sights in the final analysis, below. The red circle is 9 nmi diameter.

The blue triangle is the plot of the 3 LOPs listed above. For the most likely position (MLP) analysis we will want to know the lengths of the sides, 1.9 nmi, 1.7 nmi, 3.0 nmi, which can be read directly from the plot. (This triangle is less than half the size we get from a random selection of LOPs, as plotted in Figure 1.) And we need know the standard deviation of each of these sights.

One can argue about the right way to know the best variance on each sight session, but the definition of standard deviation is clear, and likely valid if the sights are truly independent of each other. This is why we encourage navigators to give the sextant knob a very good turn off of the sight after reading it, so that the next measurement is independent. Just following a star up or down with small adjustments is not a good way to get independent measurements.

The standard deviation is the square root of the sun of the squares of the difference between individual values and the mean value, divided by one less than the total number of sights. If you are using Excel, the function STDEV computes this value for a list of measurements.

Using that we get these standard deviations (sigmas) for the three bodies (advanced and slope-fitted)

Jupiter: sigma = 0.6 nmi
Vega: sigma = 0.6 nmi
Altair: sigma = 0.9 nmi

Without pursuing the validity of the standard deviation for such sights, these are indeed reasonable values for good cel nav sextant sights.

And now after that outline of the process,
we get to the main point of this background information... 

Once we have the best triangle we can come up with, where is our most likely position (MLP) within or near that triangle? This is the same question we would have if these were three compass bearing lines, or any other triangle of three LOPs. It is a fundamental question in marine navigation.

And that is what we have a new solution for, which is noteworthy because we believe we have treated the standard deviations and fixed errors correctly, and we can also formulate the solution in a way that is easy to compute manually underway with a simple calculator.

The manual approach is crucial for marine navigation dependability, but we also offer a free Windows app that computes the answer directly, based on entering just the three sides and three sigmas.  The formulas can also be easily incorporated into spreadsheet or a programmable calculator.

In addition to that, we have a free graphic app (to be released shortly) that lets users vary the triangle and sigmas to study how these affect the  MLP. This tool includes the addition of a fixed error that applies to each sight. Once you choose to add a fixed error, then the direction of the LOPs makes a difference, thus you see below the graphic solution has arrows on the LOPs. For cel nav sights, these arrows are perpendicular to the azimuths of the sights.

Figure 5

The graphic image above is set to closely match the example above. These sights had no known fixed errors, so the arrows do not matter. With this tool you can drag the points around to match your triangle and then experiment with the sigmas and fixed error. The light colored lines either side of the LOPs mark the extent of the sigmas you entered.  You can sometimes tell from this analysis if your data requires a fixed error to be consistent with your choice of sigmas. The ellipses mark the 50% and 90% confidence levels, discussed in the other article.

There is also a work form you can download and use to solve for the MLP by hand. A section of the form is shown below.  There are 5 solutions per page, each showing a numerical example and ways to define the triangles, which is needed because a purely manual solution requires the navigator to measure the location of corner Q3 (x,y) relative to Q1 (0,0), in addition to measuring the 3 sides, and assigning 3 sigmas.

Figure 6

Once the 3 LOPs are plotted on your chart, it should take just a couple minutes to measure what is needed and fill out the form to find the MLP, given relative to Q1. The orientation of the triangle does not matter, and it does not matter which corner you call Q1. The diagrams show the labeling once you choose Q1.

Below is a manual solution compared to our digital solution which is part of the free MLP app.

Figure 7

The top is a spread sheet solution to the manual computation that can be done with a calculator. I do not get precisely the same Px and Py manually as when computed, due to the precision of reading the values from the chart, but they are close. Also it is just a coincidence that Px happens to nearly equal the triangle side "s3" in this example, within the precision used.

With a purely manual solution we must measure location of Q3 (x=2.5, y=1.5), but this is not needed for the digital solution with the app. With the app or a spreadsheet you just enter sides and sigmas, and the solution takes seconds, not minutes.

Below is the plot used to measure Q3 and to plot the resulting MLP.

Figure 8

We should have the main note on this solution to MLP online shortly (this week I hope), with an outline of the derivation and a link to the graphic app. If you have sight data available with enough measurements or other ways to assign the sigmas then you can practice applying this. As explained in the other note, if the sigmas are all the same and there are no fixed errors, the MLP reduces to what is called the symmedian point, which is known by some navigators, but rarely used. The interesting behavior shows up when these sigmas are not the same, and when there is a fixed error folded in as well. The formalism we have is easy to incorporate into any computed solution, and indeed can be solved by hand if needed.

_____________

Please send us your thoughts, suggestions, experience, etc with this solution to MLP. They will be much appreciated.
_____________

Download MLP form.

Download MLP.exe   Computes MLP from 3 sides and 3 sigmas.

The Mac and PC versions of the interactive graphic solution should be available shortly.

100-foot Waves Expected near Aleutian Islands

There is a monster storm in progress in the Bering Strait headed to the Aleutians that promises to generate extremely high waves today and tomorrow. The unusual shape to the Low is a partial result of two storms coming together, which resulted in this long fetch. An isolated system would be more round, with shorter fetch.

By tomorrow about 06z the wave forecasts are for 60 ft, significant wave height of combined seas (SWH).

WW3 model forecast displayed in LuckGrib.  The color bar is for wave height in meters. Light blue is 17-18m (60 ft), Pink is 16m, red 14, yellow 10m

We see SWH of combined seas of 60 ft, but strangely this appears to be mostly all wind waves. The swell component in that region at the time is very low... in fact the WW3 model does not give swell data in the high wave region at all.  They just give combined seas and wind waves, and throughout that region the wind wave heights are within a few feet of the combined seas heights. The forecasted hurricane force winds are completely dominating the sea state.  Swell directions along the perimeter of the system are in all directions. This storm will indeed generate huge swells for other places later on, but at the actual storm site the prevailing swell seems to be very low.

SWH is the average of the highest one third of all waves in a statistical distribution. Other wave heights in that distribution are given in this table from our text Modern Marine Weather.

With SWH at 60, we expect the average of the highest 10% to be about 78 ft and 1 in 2,000 to be 120 ft.  These waves have a period of about 17 seconds, so 2,000 x 17 = 34,000 seconds, which is about 9 hours.  So very roughly every 9 hours or so a region would have a wave of 120 ft and it would not be considered a rogue wave. It is just at the far edge of the expected distribution of wave heights.

But there remains a valid question of where does the base 60 ft wave height come from?  Note too that an average period of 17 seconds is more typical of swells than wind waves, but these are very big waves and this is indeed a normal period for these huge waves. In fact, the waves themselves are consistent with this wind pattern which has had a fetch of some 700 nmi for a day or more.  These stats are compiled in the diagram below.

The winds have been 60+ for a long time, so just follow the 65 kt line across the diagram to a fetch of 700 or so and you see 60 ft waves with a period of 17 seconds... also note that the duration is 36 hr, all consistent with the present system.

So, in short, we get big waves, as would be expected in this system.  A few pics below present other specifications of the system.

This pic shows the swell direction on the edges and lack of swell ht forecast in the system.  The red line is 863 nmi long, and this storm needs only 700 or so to fully develop these seas.

Below are images from the WW3 site, showing period and direction of peak wave energy, SWH of wind waves, and wind wave direction and period. These can be seen by googling "NCEP model guidance" (to find this page mag.ncep.noaa.gov/model-guidance-model-area.php), then choose NPAC (North Pacific) and WW3 model, then choose time frame, and product.

Another valuable presentation of even more WW3 parameters is at polar.ncep.noaa.gov/waves.

Note the red patch of 18-sec period at the storm are waves, but the red 18 seconds off of Baja and farther to the SE are swells.  See below to note there are no wind waves in the Baja region... and the ones that are there farther offshore are going opposite directions, likely remnants of a front that is long past.

Here is an ASCAT satellite pass measuring true winds of 50 to 66 kts over a 500 mile swath at about 4 PM PDT today.  Viewed in LuckGrib.  This is a big system! Not clear how far it extends to the west.  Watch the ASCAT data online to see real values tomorrow.  LuckGrib can show these in GRIB format, as can the Ocens Grib Explorer Pro for iPad.

________________

Here is a follow up for the next day, Thursday, Oct 26.  Reports of measurements of SWH 57.8 ft.  Nice to see that science works!  This gcaptain report mixes up terminology a bit, but actual values will be known better later on.

Magnetic Dip and Zone Balancing

Traveling from Seattle to the South Pacific?

To preserve compass accuracy when traveling large distances, the dial card in your compass should be balanced in order to compensate for the dip caused by the Earth's magnetic field. The picture below is the  back of a compass card, showing the two magnets that make it work, along with the daubs of solder on the corners that are applied to balance the card.

Here are a few related notes from the Ritchie Compass company

"Ritchie compasses come standard balanced for Zone 1, which essentially includes all of the Northern hemisphere. If you're requesting balancing for Zones 2-7, simply indicate the zone that is most central to your boating area."

"Once your compass is balanced for a specific zone, it will maintain accuracy for one Zone north or south. Ritchie recommends using a compass that is balanced for the zone where the boat will be operated most frequently."

To help you judge this more specifically, see this global plot of the dip angle (inclination).

You can also look at just the vertical component of the magnetic field as another way to judge this. A high res pdf of that, along with one of the inclination shown above, plus world variation and other useful tools is at NOAA's World Geomagnetic Data Center, which is now part of the NCEI, see especially the link to these types of images (they are high res pdfs). It could be valuable to save the pdf of world variation—they call it "declination."

Here is another company's plot of the zone balancing overlaid on one of these dip maps... at some (unknown) date in the past. The lines have slipped north somewhat over whatever time period is reflected here.