Analysis of a Celestial Navigation Sight Session

This note is background for another article to be posted shortly by Richard Rice and me entitled "Most Likely Position (MLP) from Three LOPs."  We use the sights below as one example in that article, but do not include these details that must precede an optimized cel nav fix. We go over this process here and present a free Windows app for computing MLP along with a work form for solving for it manually.

We are analyzing a sight session from the book Hawaii by Sextant (HBS), which is a record of the last voyage I did by pure cel nav.  It was in July 1982, Victoria, BC to Maui, HI. No electronics at all except RDF, and that did not work well. The lack of electronic navigation was not a choice willfully made; there simply were no options at the time, which in light of how far we have come, was not that long ago. There are many sight sessions in the book; all are analyzed in detail. It is set up as an exercise in ocean navigation that is intended to be used as a training tool to master skills in cel nav and ocean navigation in general.  When you have nothing but cel nav to go by, it is important to do the best you can with each sight, and to maintain good logbook procedures.  This of course remains true today.

Here is a picture of how that fix might show up on a plotting sheet of the DR track. We are looking into Jupiter-Vega-Altair fix at Log 2614. And indeed—in light of the present topic—we do not have this fix plotted in the best possible position. In the actual voyage we just took a center value of the intersecting LOPs as the fix. Most star sights were pretty good, so this was not an issue at the time, but now we want to concentrate on doing best possible analysis of a single sight session. This process can be helpful when sights are limited or in question, and it is the only fair way to evaluate what we can and cannot achieve with cel nav.  This type of extra work is not often required in mid ocean, but could be valuable on approaches.

Figure 1

(In the book HBS, the fixes along the DR track as shown above were not made from the optimum choice of sights. Each body, as seen below, had several sights, and it is the task of the navigator to analyze these to come up with the best representative of the full set. That process is left as an exercise in the book, with detailed instructions and many examples. The sights used for the sample plots (such as the one above) were selected almost randomly from the sights taken, with the hopes that the readers strive to improve the choices and overall navigation. In short, just one sight of each body was selected, so it had no benefit at all of the others taken. The plots included are just intended to show what the layout of the DR track would look like—although the plotting is accurate, once the choices were made.)

[ Note in passing, since it shows on the plot sheet: The narrow running fix at 1227 was clearly not a very good one as plotted here, but it got corrected with a long LAN measurement with plenty of sights on both sides of LAN to get a good fix. Had we just carried on with good DR we would have been at the 2240 location fairly well. On the other hand, that running fix was not as bad as shown in this plot if it had been analyzed more carefully. The plots in the book, again, just take two random sights from each session to show a full work form and plot. We encourage readers to improve on what is shown, given the full set of data. ]

The actual 1982 navigation logbook page of the log 2614 sight session showing analysis done at the time is below.

Figure 2

First some background notes.  The actual sight reduction at the time was done with an early version of the HP-41 nav calculator that we added a few of our own functions to. With this type of sight reduction we do not use an assumed position, but do all reductions from a common DR position, given at the top of the page.

The vessel was moving at 7.3 kts on course 227 T. The average speed during the sight session, which lasted 41 minutes, was figured by subtracting log readings taken before and after the session. A standard procedure when preparing to advance all sights to a common time.

The altitude intercepts (a-values) in black pencil were done without advancing the individual lines to a common time.  The red pencil are the advanced values, which were figured from a = a + D*Cos(C-Zn), which is a mathematical way to advance the sights explained in the book. They are all advanced to the time of the last sight at 2240 watch time.

It appears that the last Vega sight was discounted at the time because it was so far off the others. In retrospect, that was a mistake!  If you advance that one you get a = 3.6 A, which is about the same as one that I did keep at the time.  (Below we see describe the argument used in HBS for throwing both of them out, but that was not applied to this session when underway.)

For the first step in this analysis now we use all of the sights.  It is crucial that they be advanced. The fix in this case will be wrong by a couple miles if that is not done.  It is taken for granted that all sights must be advanced to a common time.

When the sights are advanced to a common time, we can (as a first approximation) just average them to get an average LOP for each body... assuming the Zn has not changed more than a degree, which is the case here.

So with that averaging of the red ones we get:

Jupiter: a = 3.0' A 200
Vega: a = 3.0' A 058
Altair: a = 5.1' A 090

Note that including the last Vega sight we threw out underway changed the a-value from 2.8 (in the logbook) to 3.0.

These sights are shown below as the light purple lines. The blue ones actually plotted were from our fit-slope analysis, described below. This is an improvement over simple averaging.

Figure 3

Some  background on this plotting:  When doing sight reduction by calculator from a common DR position, the plotting is very easy. We just make a plotting sheet centered on the DR, expanded as needed. In this case, what is normally taken as 60 nmi between parallels, we just change to 6 nmi between parallels. This plotting sheet section shown is about 12 nmi square. Then the plotting is very fast an accurate, all done from the center of the compass rose, with a scale marked in tenths of a mile.

In HBS, we explain what we call the fit-slope method. That is a way to decide from a set of sights which ones are the most consistent with the known slope of the star height versus time. In other words, for five sights taken from five different DR positions (the boat is moving), we can calculate what the heights should be if were exactly at those positions. We don't expect to get exactly those values as we do not know if we were on that track or not, but the slope of these computed sights will be same for anywhere near those locations. To find the slope, we don't need to calculate all of them; we just need a calculation of height from a convenient time before and after the sight session.

When we apply that analysis we get more insight into which sights were likely good, and more to the point, which ones were likely not as good. If we have 3 sights that increased at the right slope and one that was notably off of it, then we can throw that one out and average the rest. Or sometimes we get one that if off and throw it out, and then the others scatter above an below a line at the right slope, so we can average those, or better still, take one that is right on the line.  Doing this we get the a-values below, which are plotted as the blue triangle above. We get a smaller triangle at a slightly different location. That alone is not justification for the method, which has a sounder analytical basis. We encourage anyone who wants to improve their cel nav accuracy to look into the fit-slope method, explained in detail with many examples in HBS.

Jupiter: a = 2.7' A 200
Vega: a = 2.6' A 058
Altair: a = 4.7' A 090

The fit-slope analysis almost always improves the sights. It is effectively a more logical way to do the averaging of the sights. It also demonstrates why it is so crucial to take 4 or 5 sights of each star. It is far better to take 4 or 5 sights of three well positioned stars that it is to take 1 or 2 sights of 10 different stars.

Below, for example, shows all the sights plotted (the red a-values in the logbook), with the fit-slope choices of LOPs now marked in light blue.

Figure 4.

It seems one could do some sort of filtering on this display alone, but the results of the fit-slope method do not always match what we might conclude from such a plot of all sights.   In short, you cannot tell by the spread of the sights alone, which are the good ones. However, we will indeed use the spread (standard deviation) of these multiple sights in the final analysis, below. The red circle is 9 nmi diameter.

The blue triangle is the plot of the 3 LOPs listed above. For the most likely position (MLP) analysis we will want to know the lengths of the sides, 1.9 nmi, 1.7 nmi, 3.0 nmi, which can be read directly from the plot. (This triangle is less than half the size we get from a random selection of LOPs, as plotted in Figure 1.) And we need know the standard deviation of each of these sights.

One can argue about the right way to know the best variance on each sight session, but the definition of standard deviation is clear, and likely valid if the sights are truly independent of each other. This is why we encourage navigators to give the sextant knob a very good turn off of the sight after reading it, so that the next measurement is independent. Just following a star up or down with small adjustments is not a good way to get independent measurements.

The standard deviation is the square root of the sun of the squares of the difference between individual values and the mean value, divided by one less than the total number of sights. If you are using Excel, the function STDEV computes this value for a list of measurements.

Using that we get these standard deviations (sigmas) for the three bodies (advanced and slope-fitted)

Jupiter: sigma = 0.6 nmi
Vega: sigma = 0.6 nmi
Altair: sigma = 0.9 nmi

Without pursuing the validity of the standard deviation for such sights, these are indeed reasonable values for good cel nav sextant sights.

And now after that outline of the process,
we get to the main point of this background information... 

Once we have the best triangle we can come up with, where is our most likely position (MLP) within or near that triangle? This is the same question we would have if these were three compass bearing lines, or any other triangle of three LOPs. It is a fundamental question in marine navigation.

And that is what we have a new solution for, which is noteworthy because we believe we have treated the standard deviations and fixed errors correctly, and we can also formulate the solution in a way that is easy to compute manually underway with a simple calculator.

The manual approach is crucial for marine navigation dependability, but we also offer a free Windows app that computes the answer directly, based on entering just the three sides and three sigmas.  The formulas can also be easily incorporated into spreadsheet or a programmable calculator.

In addition to that, we have a free graphic app (to be released shortly) that lets users vary the triangle and sigmas to study how these affect the  MLP. This tool includes the addition of a fixed error that applies to each sight. Once you choose to add a fixed error, then the direction of the LOPs makes a difference, thus you see below the graphic solution has arrows on the LOPs. For cel nav sights, these arrows are perpendicular to the azimuths of the sights.

Figure 5

The graphic image above is set to closely match the example above. These sights had no known fixed errors, so the arrows do not matter. With this tool you can drag the points around to match your triangle and then experiment with the sigmas and fixed error. The light colored lines either side of the LOPs mark the extent of the sigmas you entered.  You can sometimes tell from this analysis if your data requires a fixed error to be consistent with your choice of sigmas. The ellipses mark the 50% and 90% confidence levels, discussed in the other article.

There is also a work form you can download and use to solve for the MLP by hand. A section of the form is shown below.  There are 5 solutions per page, each showing a numerical example and ways to define the triangles, which is needed because a purely manual solution requires the navigator to measure the location of corner Q3 (x,y) relative to Q1 (0,0), in addition to measuring the 3 sides, and assigning 3 sigmas.

Figure 6

Once the 3 LOPs are plotted on your chart, it should take just a couple minutes to measure what is needed and fill out the form to find the MLP, given relative to Q1. The orientation of the triangle does not matter, and it does not matter which corner you call Q1. The diagrams show the labeling once you choose Q1.

Below is a manual solution compared to our digital solution which is part of the free MLP app.

Figure 7

The top is a spread sheet solution to the manual computation that can be done with a calculator. I do not get precisely the same Px and Py manually as when computed, due to the precision of reading the values from the chart, but they are close. Also it is just a coincidence that Px happens to nearly equal the triangle side "s3" in this example, within the precision used.

With a purely manual solution we must measure location of Q3 (x=2.5, y=1.5), but this is not needed for the digital solution with the app. With the app or a spreadsheet you just enter sides and sigmas, and the solution takes seconds, not minutes.

Below is the plot used to measure Q3 and to plot the resulting MLP.

Figure 8

We should have the main note on this solution to MLP online shortly (this week I hope), with an outline of the derivation and a link to the graphic app. If you have sight data available with enough measurements or other ways to assign the sigmas then you can practice applying this. As explained in the other note, if the sigmas are all the same and there are no fixed errors, the MLP reduces to what is called the symmedian point, which is known by some navigators, but rarely used. The interesting behavior shows up when these sigmas are not the same, and when there is a fixed error folded in as well. The formalism we have is easy to incorporate into any computed solution, and indeed can be solved by hand if needed.

_____________

Please send us your thoughts, suggestions, experience, etc with this solution to MLP. They will be much appreciated.
_____________

Download MLP form.

Download MLP.exe   Computes MLP from 3 sides and 3 sigmas.

The Mac and PC versions of the interactive graphic solution should be available shortly.