How the Fit-Slope Method Works

Since we first proposed the Fit-slope method many years ago, and have used it continually since then, I have just realized that the base reason the procedure works is not as obvious as we might have implied in various descriptions of it. The procedure does often lead to improved sight data; it is mentioned in the latest edition of Bowditch: American Practical Navigator. 

A look at basic least squares fit analysis forces us to find the subtlety that makes it such an important aid to celestial sight analysis. But first, a bit of background.

See also our new Fist-slope tool to expedite the analysis

__________

In celestial navigation, we take a series of sextant sights of a specific body whose height is changing with time, so we end up with a list of times (x) and sextant heights (y). Over a short period of time, the celestial body height changes in a linear way. But our measurements always include some random errors, along with possible systematic errors, as well as periodic blunders. The data points then scatter about a straight line that represents the rise or fall of the height with time as perceived from our moving vessel.

Random errors come from varying judgements on best alignment of the body and the horizon as well as random errors in reading and recording the time of the alignment. Systematic errors might be due to a fading horizon during an evening twilight session. Blunders happen by reading the micrometer incorrectly, or reading 12 on the sextant dial or watch, and then writing down 21.

Such sights might be for an LOP from any celestial body, or they could be a series of lunar distance measurements. The process and initial analysis we discuss here is the same.

We take a series of sights of any one body with the goal of getting just one value, a single time and height that best represents the series of measurements, hopefully averaging out at least the random errors, and catching and removing the blunders. An important procedure we follow to improve on this is to try to take at least 4 sights of 3 different bodies, as near to 120º apart as possible, and we alternate the sights, rather than take all of one body, then all of the next. This method also ensures we get a fix as soon as possible, in case we are somehow interrupted.

An exception occurs with evening sights of Venus, because we can often get a whole series of Venus sights with a sharp bright horizon before we can even see the stars we want to combine with it.

We end up with 4 or 5 sights of each body, and do the following for each set. We plot the Hs (y) vs. time (x) and then draw a straight line through the points, which is intended to represent what the values would be if we did them all precisely. Then we can choose any point on this line to represent the full set of measurements of that body. 

Doing this by hand, we lay a clear plastic ruler on the data so as to go through the most points possible, adjusting the line to leave as many points above as below the line. The difference between any data point and the line is called the residual, with plus above and minus below. The goal is have the sum of these residuals near zero. Points notably off the line could be ignored or weighed less when choosing the line's position. Note we can move the line up and down and we can rotate it to get the best alignment. 

This process is essentially a manual "least squares fit" to a straight line. Using a spreadsheet program like Excel, we would plot the data and then add a linear trend line, which gives us the equation of the best fit line, y = a + bx, where b is the slope and a is the intercept, meaning value of y when x = 0.

Knowing this mathematically correct best-fit line (we get from Excel), we can compare it with what we get when doing it by eye, no math involved. And it turns out that with a bit of practice, the human eye can do a remarkably good job in choosing the best fit, and thus we do not really add much to the analysis with Excel—assuming that is all we wanted out of Excel: the best fit line so we could choose a single sight that we could consider an average of the set for that body.

I think of the above as standard procedure in cel nav, without introducing the Fit-slope Method, which we will look at now.

The principle behind the Fit-slope Method is simple: we can compute the slope of the Hs vs. time line, so when fitting the line, with Excel or by eye with a parallel plotter, we can keep the slope constant as we look for the best fit. That restricts the choice of best fit line position and hence our choice of best representative sight. 

We compute the slope by computing Hc1 at a time at the start of the sight session from the DR at that time (t1) and then again from the DR at the end of the sight session (Hc2 at t2). The slope of the line is then (Hc2-Hc1)/(t2-t1). Plotting by hand we do not need this numeric value, we just plot Hc2 and Hc1 on the same plot as the actual Hs sights and draw a line between them. These Hc values will need a new scale, because they differ from the Hs values by the sum of the altitude corrections. Once on the same plot, we just align a roller plotter with the computed slope and then move it up and down, without rotation, to find the best fit.

That is pretty much the extent of the explanation we have given on how to use the method, followed usually by a couple examples. The Fit-slope Method almost always leads to an improved fix from a set of sights.

____________

But if we look at the details of least squares fitting, we can see that the above discussion has not focused in on why the method actually gets us better averages. It is implied, but not stated, and now I want to clarify how this works

I will state the key points, then at the end show the related math derivations. 

As mentioned above, a linear least squares fit to a table of x vs. y values gives the equation of the best line as y = a + bx. That best-fit line will always pass through the mid point of each axis (x-bar, y-bar), which is called the centroid point of the data set. x-bar is the average of all the sight times and y-bar is the average of all the Hs values.

Thus if the whole goal of the process is to find a single point on the best fit line using all of the sights, then you do not have to plot anything at all. Just choose Hs (best) = the average of all Hs values, and assume this is the value at time T = the average of all the sight times. That collapses the entire analysis to just averaging all of the sight data you have.

A less obvious result to many—though mathematically even simpler—is, if you force the least squares fit to have a fixed, specific slope (i.e., constant b) then the best-fit line will also go through the same centroid point of the data set, and you could again choose the midpoints as best representative of the set. The Hs values at any other time than the centroid will be different, but if the goal is just find a single point on the best fit line, the centroid point will still do the job.

According to that analysis, there is no difference between fit-slope and no fit-slope on choosing a single sight to represent the best average of the set!

The subtlety underlying why the fit-sope gets us a better average in light of the above mathematical facts is that we have assumed we are averaging all of the sights in both cases. 

In most cases, however, we are not using the same set of data in both cases. When using the proper slope we can better tell which sights are likely wrong or outliers, and remove them from the computation of the average. When we change the sights in the average, the average changes, so there is not in practice a common sight solution to the fit-slope vs. no-fit slope solution. Removing only one sight in the set can have a notable effect on the fitted line and centroid value. 

In the end, the fit-slope method is not so much a mathematical result. It is simply a guide to helping the human navigator choose which sights belong in the full set to be averaged. It is easy to find sets of real data that show that using the right slope changes the sights we consider too high or too low. 

In good conditions, we are looking for small corrections to what we might consider a good set of sights to begin with. In poor conditions, with hit and miss sights, of say a sun popping in and out of the clouds, or sights in big seas where we have to wait for a good view of the real horizon, then we might have a large scatter in the data points and the fit slope method is a hope to pull an improved fix out of poor data.

Note that fitting with fixed slope (b) in Excel is not as easy as just "Add trend line" that fits both a and b, but as noted we can do this all manually just as fast or faster than with Excel.

Another note is for the fit-slope method, we must indeed compute the Hc at the beginning and end using some app or formula to get the right slope. Pub 229 does not work for this as we need values for specific DR positions and those corrections in Pub 229 are tedious, and Pub 249 is not accurate enough.

___________

Here is Claude's proof that the best fit line goes through the centroid without a fixed slope and also with a fixed slope.

Below is then the same proof for a fixed slope:

See numerical examples in the Fit-slope tool app.

Lunar Distance Revisited with a 1939 C.Plath Kriegsmarine Sextant

We happen to have on consignment a C Plath sextant made for the Kriegsmarine in 1939 Germany. It is a prized collector's item, but we wanted to demonstrate that it remains a top of the line instrument for practical celestial navigation, and likely will for another generation or two, and there is no better way to do this than to show it can be used for lunar distance measurements, considered the epitome of sextant sights.

These sights require measuring sextant angles and index correction to within 0.1' of arc, which is the practical limit for marine sextant sights. Not having done this for a while, I was reminded of several tips that we outline elsewhere, and will note again here. The distance measured was between the moon and Venus on two consecutive nights. We will see that the moon moves to the east of the stars and planets by about 12º per night (360º/30 days).

I did two sets of sights. One with the 4x40 scope that is stock for the instrument and one set where I replaced that with a 6x30 monocular scope (made by the modern C Plath company), which is preferred for this measurement, because it makes the edge of the moon sharper. The spread in the data were smaller with the 6x30 scope, but the resulting UTC and Lon found from the measurements was actually better with the 4x40. That was just an accident, as there is always some luck involved with these sights. The higher power scope should in the longer run get better results.

This early C Plath sextant is ideal for lunars in that it is high precision and very light weight (2 lb 10z), being made from an aluminum alloy that C Plath pioneered for sextant manufacture in that era. Lunar sights take longer than conventional sights, which reminded me of the ludicrous presumptions we periodically see in advertising claiming heavy brass sextants weighing almost twice as much are preferred for their inertia and stability doing sights! Those sextants weigh more than a half a gallon of milk. Hold a full milk carton up to your nose with your head leaned back a bit for a minute or so to get the picture. The lighter sextant is always preferred and always the top of line in sextants. 

For these sights, we even want more support if we can rig it. With an eye cup that lets us press the head against the sextant, and ideally a support to lean your elbow on. Then tune the scope to show the moon's edge as sharp as possible. The star or planet, however, will always be a point of light... in principle. In fact, that point of light will have a fuzzy halo around it when looking as close as we can, and that puts a limit on the accuracy. This could be optics or it could be we just needed some eye drops!

Here are results of the first set of sights taken from a pocket beach on Puget Sound (47º 40.5'N, 122º 24.5'W), 0.4 nmi due west of our office. These sights used the stock 4x40 scope from 1939. 

These were typed into Excel then plotted, and then fit with a straight line, which we can add with a button click, called "Add trend line." This produces a least squares fit of the data, with the equation showing.

On the other hand, we can just plot the data manually and then do the fit by eye, lay a transparent ruler across the plot such that it goes though or near the most points, leaving as many below as above the line. If any are notably far from the line, just remove them, as likely in error. The goal is remove the blunders and fit the random errors.

One way or the other, however, it is crucial to make the plot to find the best representative of the full set. One or two sights alone are not enough for this process, nor in fact for traditional cel nav sights.

Note that in the analysis, we do not have to take values that we actually measured. In principle we should take a value that is on the line. Ideally we would have at least one sight right on the line and we could use that one, but not in this case.

Many lunar clearing procedures (how we get measured time and longitude from these sights) do not work well for Venus or Mars (the two closest planets) nor for LD less than 10º or so, but the Frank Reed app online at fer3.com should cover these, which is what we used. (Sample output screen at the end here.)

Here an easy first choice might be to use a LD= 4º 8.0' at 10:01:56 PM PDT 6/17 = 05:01:56 UTC 6/18. That measurement (#1) leads to a discovered Lon error of 3.3' as shown in the list below, which corresponds to a time error of 13s.

Since we know the equation for the fit to the data, we can do maybe better since our choice was not exactly on the line. If we use the equation to find the precise time (#2) corresponding to 4º 8.0' it would be 10:02:11 PDT or time error of just 3s. But both analyses have to be considered on the fortuitous side. Generally it is considered good lunar work to find the correct time to within 30s, and these measurements, no matter how careful, can yield even larger errors.

For example, if we remove the 3rd and 4th sights, which are off the most from the fit, we get a new best fit line, and that yields a 4º 8.0' LD of 10:02:29, shown below as #3, which now has the best looking data fit, but the Lon and time errors are larger.

#    UTC          LD            LD error   Lon error    Time error    Position error 

1    050156    4º  8.0'     +0.1'           3.3'             13s                    2.2 nmi

2   050211     4º 8.0'      -0.02'         0.7'               3s                   0.4 nmi    

3   050229    4º 8.0'      -0.2'           5.4'             22s                    3.6 nmi 

The main goal at hand for these sights were to show that right out of the box, the 1939 C Plath sextant we have on sale could do high precision lunars. These very good results, however, must still be considered fortuitous, especially since the index correction measurement was not as precise as we would want. It was set to zero using a direct view of Venus, which was not as sharp as it could be with more work. In the next set of sights, the IC was readjusted and measured more carefully with the solar index method.

________

For the second set, the next night, I changed to 4x40 to 6x30 and got these results for 6/18 PDT. The IC was 1.2' on the scale (see notes at the end here)

We see slightly better on the data scatter, and again if we were to take just one it would likely be the middle one right on the linear fit to the line, that leads to sight #4, which was not as good as the earlier session, though it seems the sights were better. In sight #4, the measured LD of 17º 37.7' was corrected for the IC ("If it's on; take it off!").

#    UTC          LD            LD error   Lon error    Time error    Position error 

4    054354    17º  36.5'     -0.5'           14.1'             56s                    14.1 nmi

5    054405    17º  36.8'    -0.3'             8.1              32s                    5.4 nmi  

In sight #5, to get a better fit, I removed the 3rd sight, which was off the most from the line, and then got a new slope, and used that slope to compute what time would correspond to 17º 38.0', the sextant value without IC = -1.2' applied. Note that with the third sight removed, the middle sight was no longer on the best fit line.

Excel keeps track of times in terms of fractions of 24 hr, i.e., x = (38.0+376.89)/927.58 = 0.44728). This sight is more typical of practitioners like myself who do these only infrequently. Experts can average about 15s. Note that even with fairly large Lon errors, the actual position errors are not that large at mid to higher latitudes. The position error is all Lon error since we rightfully assume that we can find accurate Lat with out accurate UTC.

Here is a sample output from the fer3.com lunar clearing solution:

The index correction was measured with the solar method using the 6x30 scope the following day. This is the preferred method for best accuracy. It requires a couple custom filters, which the article explains how to make. We have an app online that does the simple math to find the SD from two solar measurements.

So the IC was 1.2' On the scale. The actual SD at the time was 15.75', so this IC is likely accurate to within a few tenths of a minute, but normally one would take more sights than just two.

Later I will try to post more notes about using Excel to analyze cel nav sights.

Polar Wave Spectra

 

This is a place holder for soon to follow instructions on the use of polar wave spectra plots for understanding the seaway and for setting up optimum routing corrections for waves, including how to generate the plots with our new, free 

 Polar Wave Spectrum Plotter app

get the spectra data at this custom link we made

NDBC Buoys with Spectra Data

Thanks to the staff at NDBC for providing notes on the computation (References).

________________________

INSTRUCTIONS

1) Enter the NDBC buoy no, such as 41010

2) That will generate the link to the spectra data on the right

3) Click the link below that line to open that page in a new tab.

4) cmd+A or ctrl+A to select the full set of data, the cmd+C or ctrl+C to copy it

5) Paste it into the space provided

6) Click plot pasted data, which would typically load 5 days, with data every 30 min. This may vary with buoy.

7) The slide bar steps through the data.

Here is a sample of what the data should look like 

Not: period (sec) = 1/frequency(Hz)

YYYY MM DD hh mm bands
freq bandwidth energy r1 r2 alpha1 alpha2
2026 06 27 22 50 46
0.033 0.005  0.000 999.00 999.00 999 999 = 30s period
0.038 0.005  0.000 999.00 999.00 999 999
0.043 0.005  0.000 999.00 999.00 999 999
0.048 0.005  0.000 999.00 999.00 999 999
0.053 0.005  0.000 999.00 999.00 999 999
0.058 0.005  0.000 999.00 999.00 999 999
0.063 0.005  0.000 999.00 999.00 999 999
0.068 0.005  0.000   0.27   0.53 120 112 = 14.7s period
0.073 0.005  0.043   0.43   0.49 116 116
0.078 0.005  0.100   0.59   0.18 104 100
0.083 0.005  0.100   0.38   0.21 116 176
0.088 0.005  0.143   0.35   0.27 112 156
0.093 0.005  0.315   0.63   0.35 108 108
0.100 0.010  0.529   0.85   0.68 108 108
0.110 0.010  0.958   0.87   0.67 124 124
0.120 0.010  1.430   0.88   0.61 120 124
0.130 0.010  1.301   0.91   0.76 128 128
0.140 0.010  0.758   0.84   0.55 124 124
0.150 0.010  1.115   0.91   0.73 112 112
0.160 0.010  0.601   0.85   0.63 128 132
0.170 0.010  0.601   0.86   0.57 128 128
0.180 0.010  0.586   0.88   0.66 128 124
0.190 0.010  0.529   0.82   0.46 132 132
0.200 0.010  0.300   0.69   0.08 144 168
0.210 0.010  0.300   0.80   0.42 136 140
0.220 0.010  0.157   0.75   0.38 128 136
0.230 0.010  0.129   0.68   0.19 152 148
0.240 0.010  0.200   0.72   0.23 136 156
0.250 0.010  0.086   0.61   0.46 124  96
0.260 0.010  0.100   0.67   0.05 140 128
0.270 0.010  0.043   0.73   0.33 128 136
0.280 0.010  0.057   0.72   0.30 132 120
0.290 0.010  0.043   0.61   0.10 156 172
0.300 0.010  0.057   0.76   0.41 156 160
0.310 0.010  0.014   0.55   0.21 132  84
0.320 0.010  0.043   0.69   0.25 160 184
0.330 0.010  0.014   0.66   0.42 160 180
0.340 0.010  0.029   0.55   0.14 144 188
0.350 0.010  0.014   0.66   0.30 148 124
0.365 0.020  0.014   0.41   0.21 164 196 = 2.7s period
0.385 0.020  0.000   0.48   0.46 208 220
0.405 0.020  0.000   0.57   0.47 144 136 
0.425 0.020  0.000 999.00 999.00 999 999
0.445 0.020  0.000 999.00 999.00 999 999
0.465 0.020  0.000 999.00 999.00 999 999
0.485 0.020  0.000 999.00 999.00 999 999

All 9s means no data for those specs. We only plot 4s to 25s.

Back soon with notes on what the plots show and how we can use them.

Periods of 3 to 8 sec are waves (global mean ~ 5s) ; periods over 10 sec are swells, with global mean of about 11s... but with large variations on ocean and season (~8 to 18s). 

Note that not all NDBC buoys have spectra data. Ones we know about are in the interactive link above.

___________________

References

NDBC Technical Document 03-01:
Nondirectional andDirectional WaveData AnalysisProcedures 

See also:  https://www.ndbc.noaa.gov/faq/measdes.shtml Section on Spectral Wave Data.

See also SOFAR Spotter Buoy Reference Manual

Notes on S-100 and the Eventual S-101 ENC Display Standard

Background

At present all ENC content and structure (internationally) are based on the IHO S-57 standard, and how these ENC should be displayed is determined by the IHO S-52 standard. ECDIS systems are required to follow the S-52 standard precisely, which does include a few display options, whereas most ECS strive to follow those standards, but they are not required to. Some apps are far from the standard, others, like qtVlm, are remarkably close to the standard.

The change from S-57 to S-101 ENC will be part of a huge change in overall electronic navigation, described as S-100, the Universal Hydrographic Data Model, which includes: 

S-101: Electronic Navigational Charts (ENCs)

S-102: Bathymetric Surface

S-104: Water Level Information

S-111: Surface Currents

S-412: Weather Overlay

Note that the S-100 based product specifications include more parameters than listed here, and indeed their item numbers do not all start S-1xx, as noted in this weather data component. Indeed, the system is intended to encompass all aspects of coastal navigation and hydrography.

 

The new system is much more GIS oriented with various new overlays. Except for the S-101 ENCs themselves, the other components of S-100 are well underway already, with tremendous promise. With digital soundings and tide heights everywhere on the chart, we are then aware of the actual water depths everywhere, which can also be forecasted continuously over time.

So far the S-412 weather products do not add to what racing sailors have been using for years with GRIB formatted model forecasts overlaid on their charts, but this will make the data more universally available to commercial and governmental vessels in the future. Unfortunately, the GRIB format we are all accustomed to, created specifically by the WMO for weather work, will be replaced in S-100 by a new h5 format, that so far none of the popular ECS apps or ECDIS can read! 

Because of the h5 format, it takes extra work to view these new data sets. We have notes on this elsewhere, along with custom conversion apps. 

But the question at hand is the charts themselves and what will govern how they should be displayed, keeping in mind that we are looking ahead, and largely just answering a question that might have come up. S-101 charts are likely to be 5 or so years out, and hopefully by that time the rest of the S-100 suite will be well underway.

Here are notes on the S-101 ENC display standards as presented at the moment.

S-101 absorbs the display standard into itself — it will not be a separate numbered standard.

Unlike the S-57 family (where S-57 = data, S-52 = display, S-58 = validation, S-63 = encryption, as four separate documents), the S-101 ENC Product Specification covers content, structure, data encoding, and metadata for S-101 ENC data, and the same specification also includes the portrayal (display) requirements for use within ECDIS. So portrayal/display rules are baked directly into S-101 as its Portrayal Catalogue, rather than living in a standalone "S-52 equivalent."  The S-101 Portrayal Catalog is analogous to the S-52 Presentation Library.

Conceptually, S-101's portrayal does the same job S-52 does today.

S-101 is similar in content to the current S-57 object catalogue and S-52 presentation library, but implements the dynamic constructs prescribed by the S-100 framework. The key structural difference: in S-101, the relationships between features, attributes, and enumerants are defined in a single feature catalogue, and the portrayal catalogue links those feature catalogue elements to their graphical representation — both built through a machine-readable registry rather than being hard-coded into ECDIS software.

Why this matters practically.

Under S-57/S-52, catalogue updates (new symbols, new display rules) are embedded in ECDIS software and can take up to five years to roll out via software updates, whereas under S-100/S-101 the feature and portrayal catalogues are versioned in a continuously-adapted registry, letting updates happen via catalogue update rather than a full software upgrade cycle. 

There is active work explicitly comparing the two. 

The IHO's S-101 Project Team has a working document titled "Allowable Differences Between S-52 and S-101 Display," and earlier project team materials describe an explicit S-52 to S-101 portrayal gap analysis, along with a list of S-52 symbols no longer required in S-101 and proposals to retire some symbol definitions — so the transition is being managed carefully, symbol-by-symbol, rather than a clean swap.

Summary.

There won't be an "S-52 for S-101" as a separate cross-referenced document the way mariners are used to. The display rules will be part of S-101 itself (its portrayal catalogue), with the additional change that the rules can be updated from within the S-101 ENC itself.

Details of the above topics and S-100 more generally are at the IHO web site.

Shapefiles for qtVlm

May 31, 2026 update: added US states and bumped this note forward,

A powerful function of qtVlm is its ability to show shapefiles in a very convenient manner. They can even be configured to include links to live data. One example is the UK shipping forecasts that we made for qtVlm some time ago, which we put here at the top of the list.  But we need a list because there are many of these floating around that can be very useful for navigation, and I am beginning to loose as many as I find. Just found a couple neat ones for the Gulf Stream, which motivated setting up this index.

(1) UK Shipping forecasts
Note this is a special type of shapefile in that Starpath has made code that lets you get live forecasts at each zone. Normally shapefiles load static data. If we want overlays that update automatically we need to have links to images or KML files.

(2) Add elevation contours to an ENC

(3) Add north and south walls to the Gulf Stream plus Add eddies with ID 

The above Navy data were typically updated every 36 hrs, but at present (4/30/25) it is nearly a month old. So we need to keep an eye on this.  There is a lot of chaos in ocean and weather data delivery these days. A sample below:

These shape files (two are loaded here) are to be overlaid onto either the RUCOOL SST images or one of the model forecasts for the current, or overlay onto the Navy Gulf Stream Analysis to annotate what they show. These eddies will coincide exactly with what are on the Analysis maps.  Note too that these shapefiles have to be downloaded  each time they are new, which is typically every 36 hr. They are identified by day of the year, ODate, ie in 2025, 90 = Mar 31.... however, as of May 4, 2025 we are seeing only erratic updating on Navy GS products, so their fate is uncertain.

(4) Up dated US Forecast zones

(5) US state borders with state ID

The zip file US_states.zip brings with it two sets of shape files. One is the outlines only where there is a tool tip showing the state, but it is not always clear which side of the boundary it refers to. The second shape file solves that problem by planting a mark at the centroid of each state with the states name.  The download brings both files into qtvlm, but you can choose to show either one or both. To show both, you need to load both individually.

Lunar Distance by Calculator — The Letcher Method

The lunar distance method is a way to find UTC by measuring the angular distance between the moon and an adjacent body (sun, star or planet).  The measurements are called "lunars." And finding UTC is equivalent to finding your longitude. Recall we can find an accurate latitude without knowing the correct time, but standard procedures requires accurate time to find longitude. Lunars are thus a way to fill in your position if you have lost accurate time—or, starting from scratch, you never had accurate time.

It is an advanced skill in cel nav, because it requires very accurate sights and special analysis. Finding UTC to an accuracy of ± 30 sec or so is considered good work. We have notes on the history and practice of the method (starpath.com/lunars), which shows that there are essentially three approaches to the solution.

The first solution is the easy one. There are numerous apps online and for sale that solve lunars. Our own StarPilot was one of the first ones.  With these apps, you can just enter your DR position, the date and best guess of the right time, the name of the body you are using, and the lunar distance. There is usually an optional input for the measured altitudes of the moon and body, but these are not needed accurately, so they can be computed from the DR position. 

Everything else is computed and the output is the right time and right longitude, keeping in mind the uncertainty of the process: namely, every 0.1' of error in the  measured distance or clearing process leads to about 12 sec of time error, which corresponds to 3' for longitude error. 

The second solution is just the opposite: do it all by books and tables, like it was done in the late 1700s to early 1800s. The only math required is adding and subtracting longish numbers, but the tables and procedures can be complicated. With that said, this is the only logical solution, in that it is waterproof and no batteries are required.

Luckily, we have a modern all-paper solution created some years ago by Starpath friend and associate the late Bruce Stark. His book Stark Tables for Clearing the Lunar Distance — And Finding Universal Time by Sextant Observation has become a modern classic in cel nav studies. Experts consider his solution superior to some of those actually used historically (see discussion in the lunars link above.) But like the historic versions, doing this all with tables takes some practice before it becomes routine.

The third solution, which is the subject at hand, is a compromise of sorts between the first two. Namely we use a set of four trig equations that we can solve on any simple trig calculator along with standard sight reduction tables and a Nautical Almanac, and we compute the solution "by hand."

To my knowledge, this method was first put together by John S. Letcher, Jr in his excellent 1977 book on cel nav called Self-Contained Celestial Navigation with H.O. 208.  He was also a major technical authority and innovator of self steering devices, as presented in his 1974 book Self Steering for Sailing Craft. Both books are rare, but still found periodically in used book sites.

Below is an outline of his method. It has been elaborated upon in the 1980 book by Shufeldt and Newcomer called The Calculator Afloat, which is online in full. 

To help learn this process, we made an app (Letcher Lunar Distance Calculator)  and spreadsheet that solves the Letcher method that can be used to double check that you are computing the terms correctly. The app includes the examples presented in both books above. There is no logic to using this for actual solutions as there are other apps that are equally, if not a bit more, accurate that require much less input. The app and spreadsheet are purely training tools.

The idea here is you practice this a few times then write this prescription in your logbook to fall back on as needed, keeping in mind that the StarPilot, which covers all aspects of ocean and inland navigation computations, includes a lunar solution if needed.

It is important to skim over the discussion in the lunars link above. The summary is:

1) We measure the distance between moon and another body , which will be edge to edge. Then we use the Almanac to find the semi diameter of the moon (and sun if using it) so we can correct the edge to edge to get D, the lunar distance center to center, at time Ts which is our best guess of the UTC. In practice we need to take several sights and plot them D vs T then do a best fit to the line and find our best estimate of D and its corresponding Ts.

2) Then we "clear" that distance, which in this case means applying two corrections to D, refraction (R) and parallax (P). These corrections require us to compute the Hc of the moon (Hm) and of the body (Hb) at time Ts, which we do by looking up their GHAs and Decs, and then compute the distances with the formulas below. 

Note if you are underway, or have good horizon for the bodies on land, you can measure these heights in the normal way, and plot them to find a fix. The Lat of that fix will be correct, but the Lon will be off by the error in Ts, which we are finding from the lunars.

3) Once we have the cleared lunar (Dc) we need to see what time the two bodies where that far apart. We do this by computing the distance a the whole hour before Ts and at the whole hour after Ts.

Note that the lunar distance is just the zenith distance (z) to the moon assuming you are standing at the GP of the body being used. Thus D = z = 90º - Hc, which we can compute using the navigation triangle formula below (we just change our a-Lat to the dec of the body and our a-Lon to the GHA of the body).

Here is how the app input looks, showing also the Almanac data we need to enter.

Here is the sample solution from the Shufeldt book.

Once we have our measured Dc, we need to find out what time were these two bodies that far apart, keeping in mind the main premise of this measurement being that the moon moves eastward relative to the other bodies in the sky at a rate of about 12º per day, which is about 2' per minute. Not much, but right at the edge of our being able to measure this with standard sextant.

We figure this by interpolation as shown below:

Then within the accuracy of this method, Tc is he right time and Tc - Ts is our watch error.  The corresponding Lon error is 15'/1 min of time error.

The app is essentially two worked out examples that you can test with your own calculator.  The spreadsheets are there for those who want to look at that to see how these equations are entered into Excel.

Note that both examples given in the app are from May 27, 1973, so to follow though each step, here is the almanac data for that date that shows where the input came from.

These data are also needed if you want to check any of the online apps, as they typically want input of raw sextant data and then they compute everything, so to use them we have to unfold the two heights to Hs values and the LD they want is edge to edge, then they make the corrections in the apps.  Convenient for use underway, but not convenient for checking the Letcher method, so let me just add that the Letcher method does indeed work (within limits noted below), and can be improved a bit more by adding easy corrections for augmentation of the moon's SD and earth oblateness.

Letcher streses that his procedure for the important refraction correction is not valid for moon or body altitudes or lunar distances less than 10º and also the method is not valid for Venus and Mars since parallax and phase corrections are not addressed. These are not severe limitations for practical use, but we should keep them in mind.