Since we first proposed the Fit-slope method many years ago, and have used it continually since then, I have just realized that the base reason the procedure works is not as obvious as we might have implied in various descriptions of it. The procedure does often lead to improved sight data; it is mentioned in the latest edition of Bowditch: American Practical Navigator.
A look at basic least squares fit analysis forces us to find the subtlety that makes it such an important aid to celestial sight analysis. But first, a bit of background.
( See also our new Fit-slope tool to expedite the analysis described at the end plus video demo. )
__________
In celestial navigation, we take a series of sextant sights of a specific body whose height is changing with time, so we end up with a list of times (x) and sextant heights (y). Over a short period of time, the celestial body height changes in a linear way. But our measurements always include some random errors, along with possible systematic errors, as well as periodic blunders. The data points then scatter about a straight line that represents the rise or fall of the height with time as perceived from our moving vessel.
Random errors come from varying judgements on best alignment of the body and the horizon as well as random errors in reading and recording the time of the alignment. Systematic errors might be due to a fading horizon during an evening twilight session. Blunders happen by reading the micrometer incorrectly, or reading 12 on the sextant dial or watch, and then writing down 21.
Such sights might be for an LOP from any celestial body, or they could be a series of lunar distance measurements. The process and initial analysis we discuss here is the same.
We take a series of sights of any one body with the goal of getting just one value, a single time and height that best represents the series of measurements, hopefully averaging out at least the random errors, and catching and removing the blunders. An important procedure we follow to improve on this is to try to take at least 4 sights of 3 different bodies, as near to 120ยบ apart as possible, and we alternate the sights, rather than take all of one body, then all of the next. This method also ensures we get a fix as soon as possible, in case we are somehow interrupted.
An exception occurs with evening sights of Venus, because we can often get a whole series of Venus sights with a sharp bright horizon before we can even see the stars we want to combine with it.
We end up with 4 or 5 sights of each body, and do the following for each set. We plot the Hs (y) vs. time (x) and then draw a straight line through the points, which is intended to represent what the values would be if we did them all precisely. Then we can choose any point on this line to represent the full set of measurements of that body.
Doing this by hand, we lay a clear plastic ruler on the data so as to go through the most points possible, adjusting the line to leave as many points above as below the line. The difference between any data point and the line is called the residual, with plus above and minus below. The goal is have the sum of these residuals near zero. Points notably off the line could be ignored or weighed less when choosing the line's position. Note we can move the line up and down and we can rotate it to get the best alignment.
This process is essentially a manual "least squares fit" to a straight line. Using a spreadsheet program like Excel, we would plot the data and then add a linear trend line, which gives us the equation of the best fit line, y = a + bx, where b is the slope and a is the intercept, meaning value of y when x = 0.
Knowing this mathematically correct best-fit line (we get from Excel), we can compare it with what we get when doing it by eye, no math involved. And it turns out that with a bit of practice, the human eye can do a remarkably good job in choosing the best fit, and thus we do not really add much to the analysis with Excel—assuming that is all we wanted out of Excel: the best fit line so we could choose a single sight that we could consider an average of the set for that body.
I think of the above as standard procedure in cel nav, without introducing the Fit-slope Method, which we will look at now.
The principle behind the Fit-slope Method is simple: we can compute the slope of the Hs vs. time line, so when fitting the line, with Excel or by eye with a parallel plotter, we can keep the slope constant as we look for the best fit. That restricts the choice of best fit line position and hence our choice of best representative sight.
We compute the slope by computing Hc1 at a time at the start of the sight session from the DR at that time (t1) and then again from the DR at the end of the sight session (Hc2 at t2). The slope of the line is then (Hc2-Hc1)/(t2-t1). Plotting by hand we do not need this numeric value, we just plot Hc2 and Hc1 on the same plot as the actual Hs sights and draw a line between them. These Hc values will need a new scale, because they differ from the Hs values by the sum of the altitude corrections. Once on the same plot, we just align a roller plotter with the computed slope and then move it up and down, without rotation, to find the best fit.
That is pretty much the extent of the explanation we have given on how to use the method, followed usually by a couple examples. The Fit-slope Method almost always leads to an improved fix from a set of sights.
____________
But if we look at the details of least squares fitting, we can see that the above discussion has not focused in on why the method actually gets us better averages. It is implied, but not stated, and now I want to clarify how this works
I will state the key points, then at the end show the related math derivations.
As mentioned above, a linear least squares fit to a table of x vs. y values gives the equation of the best line as y = a + bx. That best-fit line will always pass through the mid point of each axis (x-bar, y-bar), which is called the centroid point of the data set. x-bar is the average of all the sight times and y-bar is the average of all the Hs values.
Thus if the whole goal of the process is to find a single point on the best fit line using all of the sights, then you do not have to plot anything at all. Just choose Hs (best) = the average of all Hs values, and assume this is the value at time T = the average of all the sight times. That collapses the entire analysis to just averaging all of the sight data you have.
A less obvious result to many—though mathematically even simpler—is, if you force the least squares fit to have a fixed, specific slope (i.e., constant b) then the best-fit line will also go through the same centroid point of the data set, and you could again choose the midpoints as best representative of the set. The Hs values at any other time than the centroid will be different, but if the goal is just find a single point on the best fit line, the centroid point will still do the job.
According to that analysis, there is no difference between fit-slope and no fit-slope on choosing a single sight to represent the best average of the set!
The subtlety underlying why the fit-sope gets us a better average in light of the above mathematical facts is that we have assumed we are averaging all of the sights in both cases.
In most cases, however, we are not using the same set of data in both cases. When using the proper slope we can better tell which sights are likely wrong or outliers, and remove them from the computation of the average. When we change the sights in the average, the average changes, so there is not in practice a common sight solution to the fit-slope vs. no-fit slope solution. Removing only one sight in the set can have a notable effect on the fitted line and centroid value.
In the end, the fit-slope method is not so much a mathematical result. It is simply a guide to helping the human navigator choose which sights belong in the full set to be averaged. It is easy to find sets of real data that show that using the right slope changes the sights we consider too high or too low.
In good conditions, we are looking for small corrections to what we might consider a good set of sights to begin with. In poor conditions, with hit and miss sights, of say a sun popping in and out of the clouds, or sights in big seas where we have to wait for a good view of the real horizon, then we might have a large scatter in the data points and the fit slope method is a hope to pull an improved fix out of poor data.
Note that fitting with fixed slope (b) in Excel is not as easy as just "Add trend line" that fits both a and b, but as noted we can do this all manually just as fast or faster than with Excel.
Another note is for the fit-slope method, we must indeed compute the Hc at the beginning and end using some app or formula to get the right slope. Pub 229 does not work for this as we need values for specific DR positions and those corrections in Pub 229 are tedious, and Pub 249 is not accurate enough.
___________
Here is Claude's proof that the best fit line goes through the centroid without a fixed slope and also with a fixed slope.
Below is then the same proof for a fixed slope:
Video demo of the process
2 comments:
I found this blog post especially helpful because it makes the properties, practical use case, and advantages of the fit-slope method clearer than I had understood them from the treatments in Hawaii by Sextant and Celestial Navigation, 2nd Edition.
One way I have been thinking about the fit-slope method is to separate two different objectives: averaging a short series of sights, and detecting outliers within that series. As the blog post points out, if the only objective is to obtain a representative sight from several observations of the same body over a short interval, then both the ordinary free least-squares line and the fixed-slope line have an important property: they pass through the centroid (point of sample means) of the observations, i.e., mean watch time and mean Hs. In that case there is really no need to fit a regression at all. One can simply use mean Hs at mean watch time as the representative sight, then apply the normal altitude corrections and proceed with the sight reduction.
Regression becomes useful when the objective is to identify bad sights (that would then be removed from the sight averaging process). A free OLS regression estimates the observed altitude vs time trend from the sights themselves and then uses the residuals to show which observations depart from that trend. An additional comment that I didn’t see explained in Dr. Burch’s books I mentioned above (perhaps I missed it), is that vessel motion does not have to be explicitly removed for this purpose, provided course and speed are reasonably steady over the short interval. The observed Hs trend contains the combined effect of the body's apparent motion + vessel's motion + measurement error. The first two components are smooth components of the trend; it is the third component (measurement error) produces the residual scatter. It is the unusually large residuals that we are trying to identify.
An alternate regression approach is the fit-slope method, which can be given a useful statistical interpretation that I do not think is usually emphasized: it is essentially a form of constrained regression. In this specific case, it is restricted least squares using exact prior information. In an ordinary free OLS fit, both slope and intercept are estimated from perhaps four or five noisy sextant observations. In the fixed-slope method, a linear equality restriction is imposed on the slope. That slope is supplied independently from celestial geometry, eg Nautical Almanac data and Hc calculations at two appropriately advanced DR positions. Thus, the observations need determine only the vertical placement of the line.
If the imposed slope is correct, the restricted estimator should be more efficient. With only a handful of noisy observations, estimating the slope is an additional source of sampling variance. If celestial geometry already provides that parameter with much greater precision, estimating it again from the sights is unnecessary. The fixed-slope method therefore removes one source of estimation noise and reduces the number of parameters that must be inferred from the sextant data.
The qualification is the usual one for restricted estimation: the efficiency gain is conditional on the restriction being correct. If the imposed slope is wrong, the model is mis-specified and the restriction can introduce bias.
Conceptually, then, this is a variance vs bias/misspecification, tradeoff. Free OLS has more variance because both slope and intercept are estimated from a very small noisy sample. Fixed-slope regression reduces that variance, but in exchange relies on the externally supplied slope being sufficiently accurate.
That raises the practical question of how sensitive the Hc-derived slope is to DR error. Over a short series of sights on one body, the answer seems favorable. The slope depends on the change in Hc between the beginning and end of the sighting window. A common absolute DR error at the two endpoints affects both Hc calculations similarly and therefore largely cancels when their difference is taken. What matters much more is error in the vessel's relative movement over the time interval, ie course, speed, current, or leeway. Over a 10- or 20-minute sighting period, that relative movement should normally be known relatively accurately.
So, to restate Dr Burch’s practical distinction: If the goal is simply to average several good sights, use the centroid directly: mean Hs at mean watch time. No regression is necessary.
However, if the goal is to detect outliers, either free OLS or fixed-slope regression can be used to examine residuals from individual sights. But the fixed-slope method has an important statistical advantage: it substitutes well-determined physical information for a slope that would otherwise have to be estimated from a very small, noisy sample. Provided the celestial/DR-derived slope is sufficiently accurate, that should give a more efficient and more stable basis for identifying anomalous sights.
Post a Comment