Tuesday, November 13, 2018

Fit-Slope Method to Analyze Sextant Sights

We have been teaching what we named the Fit-slope Method for analysing sextant sights since 1978, and it has appeared in all of our textbooks since then. Consequently we were pleased and honored that this method was referenced in the latest edition of Bowditch's American Practical Navigator, which in turn led me to realize that we do not have an explanation of this method in public that is easy to reference, although I did post a note related to it.  That note, and the Bowditch description, are short, and do not emphasize the key aspect of the method. This note is intended to remedy that.

*  *  *

For any measurement, we always get a more realistic value by making several independent measurements and then averaging the results—or maybe even applying more sophisticated statistical analysis such as least squares.  This is no different in celestial navigation when we measure the sextant height of a star from which we can compute a line of position (LOP) on the chart. We do not want to rely on just one measurement, we want many, so we can average them in some manner.

The problem we have in the sextant measurement is everything is moving. The star is moving across the sky; we are moving across the water; and the clock hands are moving across the watch.  It is not like measuring the length of a board, where we could measure it 5 times and then average the results. In the case of the sextant height of a star, we expect it to be different every time we measure it, so we have to clarify what we mean by an average.

If we were not moving at all, such as sights on land or when dead in the water, then we only have to account for the rising or setting of the star. We will get a set of sextant heights (Hs) and specific watch times (WT). Below is an example of what these might look like when plotted as a graph.
Here we have 3 sights of a star that is descending in the sky. The vertical scale might be 5' per division with say 44º 20' at the bottom and 45º 05' at the top; the horizontal scale might be 30 sec per division starting at say 16h 20m 00s on the left and 16h 24m 30s on the right. To choose the scales, we look at the range of time and range of heights and compare that to the graph paper we have, and choose some convenient scale based on that. This type of data can also be plotted digitally with a spreadsheet program such as OpenOffice Calc (Windows or Mac), and some cel nav programs like our own StarPilot do these plots automatically.

Over the duration of typical sight taking sessions (20 minutes or so), most celestial sights will follow a near linear path (meaning change in height per unit change in time is constant), providing the celestial body is not crossing our meridian at the time. So our first step analysis without further information is to just draw a line through these points using a manual form of least squares fitting. Namely the line should go through as many points as we can, leaving roughly the same number above the line as below—and maybe ignoring ones that are anomalously off the line as they are likely blunders. With little practice, navigators can actually draw this  "best fit line" as well as a computer program could using a least squares analysis.

When we choose where to draw that straight line, we have two degrees of freedom. We can move the line up and down on the page, and we can tilt the line, which is equivalent to changing its slope. The dotted line above is one choice of best fit. Once a best line is chosen, we can then choose which value will represent the "average" of the sights taken. If the line goes right through one of the points, then we can use it, and that single one will represent an average of all of them.

On the other hand, if our best line does not go through one of the actual sights, we can choose any convenient point on the line and use that for the representative average, as shown in the figure below. In these cases we might choose a point at a whole minute to simplify the later analysis. In the figure above this would be using Hs1 and WT1 to represent the average of all 3 sights, even though we did not take an actual sight at that time. With 5' per vertical interval, we then have to accept that the three sights are about ± 2' off of this average.
All of this procedure described above is what we might call standard practice, and something equivalent to this is used in most cel nav programs. This standard method, however, does not take advantage of all we know about the sights, and as such it often means we are giving up accuracy in our fix that we do not need to give up.  With that in mind,  let's look at the data again.

As we shall see, having just 3 sights is not optimum; we are better off with 4 or more, but with just three it is easier to see the point at hand.  For now we assume the slope is linear, which it is in most sight sessions—and if not, we will be able to know that as we show later. With a linear slope, we can look at these sights as the first two being consistent, and the third one being off.
Or we can think of the last two being consistent and the first one being off.

Or the first and last are consistent, but the middle one is off.  The difference between these conclusions is the slope of the data, meaning how fast does the height of the star change with time as viewed from our location. This slope also depends on the motion of our boat because that changes our position, but we will come back to that detail.  For now, we just assume we are on land or dead in the water.

This brings us to the whole point of the Fit-slope Method.  This slope is not a degree of freedom we have to vary when choosing the best line. We can easily compute this slope and then force our line to have the right slope. In other words, in the three interpretations above, only one is right. We just have to take a few minutes to figure out which one it is.

To do this, we perform standard sight reductions of the body sighted at times (T1 and T2) just before and just after the sight session, using the corresponding DR positions. This tells us what the heights (calculated height, Hc) should be at these specific times. We also get the azimuth, Zn, but we do not use that. These two Hc values can be plotted on the same plot as our data using an offset scale for the heights.  The Hc heights won't line up with Hs heights because Hc and Hs differ by all the standard sextant corrections (index, dip, and altitude). Once the two Hc values are plotted, we can see what the right slope should be.

This special determination of Hc values for the slope is best done by computation rather than using sight reduction tables. This way you can get Hc from the exact DR positions you were at times T1 and T2, which is what we need for finding the most accurate slope. You can compute Hc with a trig calculator using the basic definitions of the navigational triangle, or use a program such as the free Celestial Tools by Stan Klein. If you are online you can get Hc from the USNO—use our shortcut link starpath.com/usno. I would mention using our own commercial product StarPilot, but this step is not needed in that program as it solves the Fit-slope Method automatically.

You can if needed compute the Hc with tables alone (no computers,  no calculators), but it is extra work. Most sight reduction tables in common use require employing an assumed position, which can be as much as 30 miles away from your DR positions. The resulting Hc values are perfectly good for getting an accurate LOP, but the time slope between these two Hc values at different assumed positions is not accurate enough for what we want.

Once we have the slope plotted as above we can use parallel rulers or a roller plotter to move it up into the data to find the best fit.  We are still doing a "manual least squares" fitting by eye, but now the slope is not a variable. In many cases, this helps us rule out certain sights. In this schematic example, sight C is most likely not right, and we are likely best to just average A and B and use that to represent the set of all three. When there are 4 or more sights, the confidence in choice of outliers is more satisfying—or we learn there are no obvious outliers, and we use the spread of differences from the slope line as a measure of our uncertainty in that sight.

This procedure is not a magic bullet in any sense. It is simply imposing onto the data a property that we know (the slope) and using it to help us choose which sights might be less reliable. It is not guaranteed that the conclusions will be right, especially when there are few sights and the spread in values is not large.  But this method does give us a bit more confidence in our choices.

Since we are looking for small differences, it is important when we compute the Hc values to use the right DR positions, which will change during the sight taking session. In class we teach that each sight session starts below decks with a recording in the logbook of the time, log, course, and speed.  Then after the sights are completed, we return to the logbook and record the ending time, new log, and again record course and speed. We can use that information to mark the starting and ending DR positions on the DR track that is already plotted on our plotting sheet.  We had to have that track plotted to predict the time of the sights, and we use it again to choose a DR position for the sight reductions. (See Notes on DR and Sextant Sights.)

To show the influence of this, consider we are sailing 6 kts toward the south, taking sights of a body toward the south over a period of 30 min. Bodies near due south are crossing our meridian, so their height is not changing much with time if we were not moving, but sailing toward it we would raise its sextant angle by 3' during the 30 minutes we are sailing toward it at 6 kts.

Below is an example fix using the fit-slope method. It is from the book Hawaii by Sextant, analyzed with StarPilot. Sights are from a 1982 race, Victoria, BC to Maui, HI, near the end of the race approaching land. No other navigation was available at that time but cel nav. There was a radio beacon there, but it was not helpful for RDF at the time.

Figure 1. Raw sight data of Problem 27. C=232T; S=7.6

Notes from the book: "We are getting close to sighting land. This could be our last chance for a good fix. We do a set of morning star sights. Usually in star sight sessions we add Polaris whenever we have time to do it because it takes no pre-computation. Just set your sextant to your latitude and look north. But in this case it was woven right into the set of all sights, which were alternated in time to get the best fix."

Here is what these sights look like after they have been reduced using a calculator.

Figure 2. Analysis of the sights

This analysis does not take into account the motion of the boat. They are all reduced from the DR given in the Polaris sights. Each is valid at the WTs shown.  Below we plot all of these, adjusting each sight to the time of the last sight using Speed = 7.6 kts and Course = 232 T.

Figure 3. Plot of all sights, adjusted for boat motion. In the StarPilot, the individual LOPs are identified with a mouse cursor rollover.

The light gray vertical and horizontal lines are meridians and parallels, which end up at unusual spacing in the StarPilot plots. Here the two latitude parallels are separated  by 40.9'.

Figure 4. Showing bad Capella sight is way off the slope line (22.7' off) and can be discarded—although we did not need the fit-slope method to know this one was bad.

We see immediately that one of the Capella sights is way off and certainly a blunder, leaving us to look at the other three. Likewise, Vega has one sight notably off the others.  There is not much we can do with the Polaris data as there is no significant slope to its motion. Below are a couple of the fit slope plots.

Figure 5. Capella sights. After discarding the bad sight, the last 3 have more information. Relative to the second two, which are close to the line, the first is clearly higher. In this case, however, the difference off the line is under 1.0' so we probably won't get much difference in the fix from using the last two only or all three. 

Figure 6. The 3 Vega sights. The second is off the slope of the other two by 4.1', which we had indication of from the plot of all LOPs. So for this sight, we choose the average of the first and last as likely the best average for these 3 sights. Without the known slope line, you would have the conditions discussed in the introduction, namely not know if 2 and 3 were right, leaving 1 high, or if 1 and 2 were right, leaving 3 very high. 

Now we can redo the plot of all LOPs employing the choices from Fit-slope and again correcting for vessel motion. This is shown below.

Figure 7. Plot of all sights corrected for vessel motion. Each LOP shown represents the average of multiple sights guided by the fit-slope analysis. This is Problem 27 in Hawaii by Sextant.

My experience is that this method generally improves the fixes from typical sights at sea. In some cases, the choices of best sights to use are obvious. In other cases, we have to look at the full set of sights combined with other sights to help us make a choice.  The method is likely of most value in helping set a confidence level on the fix. If we have four sights and three line up well, but one is off, but just by, say, 1.5', then in many cases we might not feel confident to throw that one out, but with the other three nicely on the line and that one off by that much, we are more confident to remove it. This operation can then leave us with a tighter fix with less uncertainty.

Our book Hawaii by Sextant is once way to practice the method as there are many real sights and fixes included. It can certainly be tested on land, and indeed used by those new to sight taking to help evaluate their sights. You can apply it to a set of sights using a shoreline on the other side of a lake, or use it with an artificial horizon. The application of the method will be good training for actual sights at sea.

To get the ultimate accuracy from a set of sights then requires us to analyze the triangle of 3 LOPs to find the best location, or most likely position, based on the shape and size of the triangle of intersections ("cocked hat"), as well as the standard deviation or variance we assign to each LOP in the set. Often these variances are the same, but when they are not, it affects the best choice. 

One of the key outcomes of the fit-slope method is a better feeling for the uncertainty in an LOP based on the analysis of the several measurements we made to get the LOP.  In the above discussion, if we did no analysis we had to assume something like ± 2' for the variance of this LOP, based on three measurements, not counting other information that might affect this value.  After the fit-slope analysis, this variance can be fairly reduced to about half of that, or better.

We have a related discussion called:  Most Likely Position from Three LOPs.

Monday, October 22, 2018

Marine Weather Workhorses... and Secret Sources

We are in the process of updating parts of our Weather Trainer software and ran across an article we wrote in 2009 that we refer to frequently. And sure enough there are parts that needed updating (which this note addresses), and surprisingly much is probably still new to many mariners. The "workhorses" have changed a bit, but the "secret source" is still more or less secret! It first appeared about sometime in 2009, but without any announcement, as far as we could tell at the time. It is hidden right under our noses—I mean mouse cursors—in the same place online many of us check every day to see what clothes to wear to work. But let’s come back to this jewel in a moment.

Once underway on inland and near coastal waters, the NOAA Weather Radio on VHF is likely to be the main workhorse for inland waters. It gives observations every 3 hours, forecasts every 6 hours, and synopses every 12 hours. The broadcasts are continuous, 24-hr a day. A typical broadcast is about 10 minutes long, which includes some inland and mountain weather.  The NWR site has many resources about each of the stations, from which you could piece together a picture like the one below that we made for our Modern Marine Weather, 3rd ed text, which has an extended discussion of this resource.

Figure 1. NOAA Weather Radio coverage on the NW coast, showing typical ranges as well as station overlaps and VHF channels. The Canadian counterpart is called Weatheradio.

But that is not the best place to start a study of what might take place on a planned voyage, even if you happen to have a VHF radio that will receive the signals at home.  All of the information in these radio reports are available as text. It is just a matter of sorting out the best way to access these text reports.

The NWS offers forecasts by region (zone) in various formats. Some of these are itemized in a publication called Marine Weather Information Guide, but we have frankly more information and perspective on the various categories of forecasts in our textbook—this Guide, for example, does not cover the coastal zone system we discuss below. There are multiple other groupings of zones in use, but the primary practical categories are coastal zones, offshore zones, and high seas areas. Coastal zones are typically in two bands, 0 to 10 nmi offshore, and 10 to 60 nmi offshore on the West Coast, and 0 to 20 and 20 to 60 off on the East and Gulf Coasts.  Bays, sounds, lakes, and estuaries are covered in the coastal zones using custom geographic definitions to match the waterways.

Web Access to Coastal Zone Forecasts 

Coastal zones are the most localized marine areas covered by official NWS forecasts—although we can look more locally as covered later (the secret!).  The link weather.gov/marine  has a link to the page that brings up the Coastal Zone graphic index shown below.

Figure 2. Index to US Coastal Zones. Click any colored region to zoom into that coast. You can also get to this index with Google to "NWS coastal zone forecasts."

 Figure 3. Coastal waters of the West Coast. 

Each color, named by a prominent city, identifies a group of coastal zones called the coastal waters Below we the coastal zones included in the group called "Seattle Coastal Waters."

Figure 4. Coastal Zones within the Seattle Coastal Waters area.  

When viewing online, we can click any one of these zones to get that local forecast. Below is the report we get from clicking anywhere in the PZZ132 zone (dark magenta, top right). This region is the Eastern end of the Strait of Juan de Fuca.

Figure 5. Web presentation of pzz132 forecast, with several related links given.

This internet approach is graphic and straightforward, with several interesting links to related information, but notice we have lost track of the name of the actual zone we asked for, which was pzz132. The notation "pzz100" at the top refers to the synopsis of the full region covered by the "Seattle coastal waters," which covers all the zones shown in Figure 4.

Coastal Zone Forecasts by Email

Individual West Coast coastal zone forecasts are stored online at NOAA at a link like this one: http://tgftp.nws.noaa.gov/data/forecasts/marine/coastal/pz/pzz132.txt  To get a different zone, just change the pzz132 to the zone you want.  All the files that are even 100s are synopsis files for various coastal waters regions. Other coasts are in other subdirectories:

/pz = West Coast
/an = Atlantic Lat > 31 N
/am = Atlantic Lat < 30 N
/ph = Hawaii
/gm = Gulf of Mexico
/pk = Alaska

You can get an individual zone forecast file with the NWS FTPmail service, or even more easily from Saildocs: send an email to query@saildocs.com that is all blank except for this one line:  send pzz132

That will get you the report in a few seconds by email. (If you want the regional synopsis as well as that specific zone forecast then add another line to the message with send pzz100.)

The email version you get back will look like:

Getting your forecasts this way requires a computer, tablet, or smartphone. Throughout the Pacific NW waters and the Salish Sea we have good cell phone connections, so this generally works well.

Coastal Zone Forecasts (and Observations) by Voice Phone

Without wireless devices and network connections, you can still get these forecasts with a flip phone or princess phone along with latest observations using a NOAA service called Dial-a-Buoy.   Call


and follow instructions. If you do not know the ID of a buoy or lighthouse near the region you care about, then choose option 2 and enter a Lat-Lon. Then select a buoy to get latest observations and following that you can request the forecast for that region, which will be the coastal zone forecast of that buoy location.

This is becoming a bit old school, but it could pay to have that number in your list of contacts as a back up if you are on a vessel when other options fail.

Coastal Zones versus Coastal Waters Forecasts

Coastal waters are groups of coastal zones. You can tell from the index shown in Figure 3 how these are grouped on the West Coast. There are similar groupings along all coasts. Below shows an example from Florida

Figure 6. Coastal waters of Florida.

Figure 7. Coastal zones included in the Miami Coastal Waters region. 

A typical file here would be found at http://tgftp.nws.noaa.gov/data/forecasts/marine/coastal/am/amz630.txt for Biscayne Bay, and the file /amz600.txt would be the synopsis for the full Miami coastal waters region shown above, which will include, by the way, the location of west wall of the Gulf Stream in this region.

Coastal Waters Forecasts by Email

For local daily sailing, we would likely care about just the coastal zone that covers our waters, but when transiting a region it could be more convenient to download the coastal waters forecasts (CWF) that includes several adjacent zones.  The file names for the CWF files are more complex, but saildocs has a short cut for requesting them.

Figure 8. Coastal waters file names from Modern Marine Weather, 3rd ed. These files (CWF) are groups of coastal zone forecasts covering about 200 nmi along the coast. They are available from Saildocs, i.e., send fzus52.ktbw for coastal waters around Tampa, FL. Use just the green part of the full file name.

Observations from NDBC

When we want near live observations rather than forecasts, then the primary source is the National Data Buoy Center (NDBC)—easy to find on Google with NDBC.  This site is very easy to use. Zoom into the buoy or station of choice and you see the results.

One subtlety we confront are the units options.  Choose either "Metric" or "English." With metric we get mb for pressure but have to live with m/s for wind speeds. Choose English and you get knots for wind speed but then stuck with inches of mercury for pressure, which is intended for aviation and TV weather.

The small graph icons on the pages are links to plot the data. Below is a unique option that shows both wind and pressure, showing the units for Metric and English.

Figure 9. Data plots from NDBC. The inside uses English units; choosing Metric units puts the outside scales on the data.

Once you know the ID of a favorite station you can find it easily on Google with that ID alone. You can also use it in the Dial-a-buoy program mentioned above. Likewise you can get the latest data including sea state by email from saildocs with this query (change the file name to the one you want):

send http://tgftp.nws.noaa.gov/data/observations/marine/latest_obs/41002.txt

NWS Mobile Weather App—Sort of

The NWS does not have an actual app, but they have a mobile website set up that you can save in your phone to access much of their data. The link to the page is

Using an iPhone, you can open this in Safari, then press the share button,  and choose save to home screen. This then is effectively an app.  A few clicks gets you to a display of the coastal waters forecasts, and another few clicks gets you to a list of buoys for latest observations. You can even view the latest weather maps, called "radiofax charts" on the menu.

AND FINALLY... The Secret Source

The secret source of NOAA's marine weather is actually just a normal part of NOAA's land weather. It remains a bit of a secret because landsman are very unlikely to use it, and mariners are very unlikely to know to look for it there.  When this first came available (maybe 10 years ago), it did not have much written about it from the NWS... but we did have our original "secret sources" article. Now we can find more official discussion, and we know more about what it really is.

In short, it is a graphic interface to a point forecast, based on the National Digital Forecast Database (NDFD). The way to access this is to ask online for NWS weather for any town next to the water you care about. Once we get close to the water, we can go from there.  For example, let's ask Google for "NWS Seattle weather."  You must include the NWS or you will be bombarded by commercial weather pages, which vary from bad to worst.  The right NWS official page will have a URL  like https://www.weather.gov/sew/.  If you asked for "NWS Chicago weather," you would get https://www.weather.gov/lot/, and so on.  That type of format is what we want.

Alternatively, you can get to the right place by searching on or going directly to

Then click the national map near where you want the marine weather. Once near the waters you want, click more specifically to get something like shown below, where we have clicked in Puget Sound, at the top edge of Elliott Bay, just inside West Point. Then in the figure we look at the forecast at another location in the Bay.

Figure 10. Point forecast for just south of WEst Point Lighthouse

The details don't matter much at this point, other than to note these are different forecasts, inside and outside of the Bay. Though not too different now, they could be notably different in other conditions.

Figure 11. Point forecast for inside Elliott Bay

Now compare both of these "point forecasts" to the official coastal zone forecast for this region (pzz135) shown below.

Figure 12. Coastal zone forecast for Puget Sound (pzz135)

These point forecasts are in principle more accurate than the coastal zone forecast that covers a larger region, which might easily be expected for inland waters like these where the terrain and shape of the waterway is so varied over the forecast region.

We can also get more granular information on other weather factors using the meteograms at the forecast points.   These are at the bottom right of any of the above weather forecast pages. They are called Hourly Weather Graphs. Below is an example showing temp and dew point just north of West Point (Figure 10).

Figure 13. Meteogram from NWS land weather page, clicked offshore, just north of West point. Note this is a forecast, not a report.

The present time (3pm PDT) is on the left side here, which shows a forecast dew point and air temp about 2º apart, which would normally be no fog.... or patchy fog. This is one of those cases where we would want the forecaster to look out the window—or at least read the reports from the lighthouse on the point shown below.  These temperatures have been the same all morning, which we can learn from the reports at West Point Lighthouse, shown below.

Figure 14. Data from WPOW1 showing dew point equal air temp all day.

It is pea soup on Puget Sound all day today. From the Figure 13 forecast, we can expect it to clear up about  9 or 10 am tomorrow, which (looking back now) it did in fact do on time.  So despite being off on Monday with regard to fog, we can see the value of this type of presentation.  Frankly, these one or two day NDFD forecasts are right way more often than wrong.  We also see an informative presentation of the wind forecasts in these meteograms, which is especially valuable for planning if you have a front coming through during a race.

Figure 15. View of Puget Sound at 3 pm PDT as I write, viewed  from about 270 ft elevation, 1.8 nmi NE of West Point. Although fog is not uncommon, this very thick fog is rare here.

With that said about the workhorse sources of local forecasting, let me add that there is more to this topic. Please refer to our textbook for details and recommendations. None of the sources covered, for example, would be considered the best possible forecasting for sailors. The best would depend on the time frame in mind—plus there is some extra work involved in accessing them.  For the next 12 hr or so, we would likely do best with the HRRR forecasts that extend out 18 hr and are updated every hour. For longer term, say three to five days, then it depends a bit on the waters we care about, but in the not too distant future, the answer will probably be the National Blend of Models (NBM) forecasts regardless of waters.

Figure 16. Cover of Modern Marine Weather, 3rd ed.  Also available in all ebook formats.

Friday, September 28, 2018

Solar Index Correction Method for Sextant Sights

This is a specialized technique in cel nav that is only needed when pursuing sights to their ultimate accuracy. It is on the exponential end of gaining an extra small bit of accuracy at a fairly high price in effort. On the other hand, with a systematic approach to the measurement and to the special tools required, it could be more widely used, with a corresponding rise in cel nav performance.
Modern celestial navigation was well underway by the end of the 18th century.  Almanacs were readily available and numerous mathematical prescriptions for doing the analysis of several sights were widely known. Newton's original invention of the sextant had pretty much replaced all earlier measuring devices, and there were several well known textbooks on navigation. A primary reference from that era was Nevil Maskelyne's Tables Requisite to be used with the Nautical Ephemeris for Finding the Latitude and Longitude at Sea.  The first edition was 1766. We have at Starpath an original of the 3rd ed from 1802. That edition is also online as a PDF.  It is mostly tables, but there is a short section we care about now called the "General Introduction Concerning Instruments and Observations," which is at the back of the book!

This note is essentially a reiteration of part of that Introduction on how to measure a sextant's index correction using the sun. It is no surprise that there is not much to add. Other than bringing it to light for those who have not tried it, we outline our proposed procedure and describe our work form to compile the results. This topic (and motivation for this note) came up in our online classroom discussion as we go to press with a full set of our Starpath Celestial Navigation Work Forms, which includes detailed instructions and worked examples. But this particular operation and form calls for more background than needed for the other forms, because it is the rare instance where celestial navigation can be dangerous! Our Forms booklet outlines the basics of this process, and then refers readers to this note where we fill in the background.

I will outline our procedure and show our form for this and follow up with the actual wording in the Tables Requisite to show that none of the key principles have changed. In the process we discuss the sight taking and how to build a custom sun filter for this measurement.

First a reminder of what we are after. For the sextant to read the proper angle, the two mirrors, index and horizon, must be strictly parallel when the sextant reads 0º 0'. The sextant comes with ways to adjust both mirrors to accomplish this, but rarely, if ever,  is it justified to try to get these set just right, because after each adjustment we must do multiple careful tests to confirm it, and the adjustments are interlinked. One adjustment screw moves the mirror mostly one way (think of it as a vessel motion roll), but at the same time also moves it a little bit the other way (which can be thought of as a vessel yaw); the other adjustment screw does just the opposite. In short, there is cross talk in these adjustments.  The best procedure is to do the best we can to get it very close with reasonable effort, and then stop, and spend the rest of our allotted time on this to carefully measure how much it is off.  Then we just correct each subsequent sextant sight for that amount. The offset is called the index error; the correction for it is called the index correction (IC). The terms are often used interchangeably, but technically they would have opposite signs... which is confusing, and we don't want confusion, so we stick with the concept of IC.

The standard way to measure the IC is to set the sextant dials to 0º 0' and then look at any distant object through the telescope. When underway, we usually look at the sea horizon, which will appear slightly split (when there is an IC needed) at the intersection of the horizon mirror (the reflected view) and horizon glass (the direct view). We then adjust the micrometer until two views of the horizon are aligned, which implies the two mirrors are now exactly parallel, and then read the dial. If we get a small reading such as 2' then this is called "On the scale," or just "2' On."  In other words, this sextant read an angle when there was none, so we have to subtract that amount (the IC) from all future sights—it is like a speedometer on a car that reads 2 mph when you are parked. When moving that speedometer will read 2 mph too high.

On the other hand, if the sea horizon alignment brought the index to the other side of 0' we would read something like 58', and then we know we are 2' "Off the scale," so our instrument will read 2' too low on future sights. All celestial navigators know the jingle on how to make the correction:  "If it is off, put it on; if on, take it off," because we do not want to gamble with ± signs for such an important correction. Below is a reminder of how these readings appear on the micrometer drum.

Figure 1. IC measurements. Left reads Hs = 0º 2.0', which is IC = 2.0' On.  Left reads Hs = 0º 58.0', which is equivalent to  Hs = – 0º 2.0' corresponding to IC = 2.0' Off.  A counterclockwise (CCW) turn of the dial is in the direction of larger numbers on the micrometer, which in the telescope view brings the reflected objects down relative to directly viewed objects.  We call this CCW direction "Away" as we are turning the top of the dial away from us.

The topic at hand is the reference object we use to align the mirrors. Instead of the sea horizon, it could be a star or planet, or it could be a limb of the moon. What is not often noted is that it could in many circumstances be a cloud.  The alignment of any of these objects (sea horizon, star, planet, moon, or cloud, or a hilltop, a couple miles off) will indeed get us a measure of the IC, but it is not as accurate as we can do by using the sun. The sun is, frankly, the most trouble to use, but it does get the most accurate results.

"...the sun is incomparably the best object for this purpose."
—Nevil Maskelyne, 1766

Furthermore, none of these other objects gives us any direct measure of the accuracy of our IC measurement, whereas using the sun does indeed give us a check on our measurement. We can use this method to measure the sun's vertical width, which in turn we can look up in the Nautical Almanac—it varies slowly throughout the year.

The sun yields the most accurate IC, but we need very special care in the use of sextant shades or custom filters to be certain we do not get any direct look or even a glance at the sun during the process. That is the dangerous part of this technique, and probably why we do not often see this in modern textbooks or classroom courses. Even with the proper shades and filters discussed below, there can be fatigue to the eyes when applying this method, because we have to look toward the bright part of the sky frequently in order to know where to point the sextant.

Below we have a schematic presentation of the principle of what we are doing, then we look at the details of the actual measurement.

Figure 2. Schematic of a solar IC measurement. The round sun is stylized to be square so we can concentrate on the top and bottom edges. Direct view through horizon glass on the left is white; the reflected view in the horizon mirror is gray. This schematic has the suns offset horizontally, corresponding to a large side error. In practice they would be overlapped. Keeping the direct view on the left centered in the telescope, as we increase the value of Hs on the dial it brings the reflected sun down in this view relative to the direct view.

Steps 1 and 2 depict a "normal" IC measurement. With the sextant set to 0º 0', a first look shows the misalignment when an IC is needed. Bring them together turning the dial in whichever direction is needed, and then read it when aligned. The top of Step 1 would yield an IC On the scale; the bottom view would yield an IC Off the scale.  Let's say our true IC is 1.6' On the scale. We might not get exactly that in Step 2, because it is not easy to judge exact overlap of two disks—keeping in mind that in the telescope view the suns are inline, not side by side as shown in Figure 2.

A better approach is the crux of this solar method. Namely we measure the vertical diameter of the reflected sun—just as we would measure the height of a star above the horizon—using first the bottom edge of the direct view sun as a horizon reference and then again using the top edge of the direct sun as the reference. This gives us two measurements of the diameter of the sun, each of which depends on the value of the IC and the sun's semidiameter (SD). With two equations and two unknowns, we can solve for both, and use the value of the SD in the Almanac as a quality check on the results.

We first use the lower limb of the direct view for a reference (Step 3), and then the upper limb (Step 4). This will give us what we call an "on scale width" and an "off scale width"—names recommended by Maskelyne in 1766.  To consider actual values we would measure and record in the form, let us assume we have an IC of 1.6' On the scale and that the SD of the sun at the moment is 16.2', which is a typical value in the range of 15.7 to 16.4.

Looking at Step 3, we started 0º 0' in Step 1, we turned the dial to bigger numbers stopping at 0º 1.6' at alignment. Then we continue to bring the reflected view down by the full width of the sun (32.4'). When the top of the reflected view is aligned with the bottom of the direct view, the dial will read Hs = 34.0' (32.4 + 1.6). This reading Hs (on) = 2*SD + IC. This we call the On value of the width of the sun, which we enter in a form later.

Step 4. Now turn the dial the other direction, which moves the reflected sun up in the telescope view. This will be a rotation in the direction of smaller numbers on the dial. As we continue, we cross back over the view of Step 2 (Hs = 1.6') and then we keep rotating toward lower numbers till we are at the Step 1 view, which will be 0.0' on the dial. To align the LL we just have to rotate backward from 0.0 by an amount (32.4 - 1.6), or 30.8'. This puts us 30.8' behind 0.0', which we read on the dial as 29.2' because the dial counts backwards in the off-scale direction.  In our forms, we will record the off value as 29.2 then subtract it from 60 to get Hs (off), the Off value of the width of the sun, and record it in the form. In this case, Hs (off) = 30.8' = 2*SD - IC.

Figure 2a. Dial readings for Hs On and Hs Off for the sun width measurements.

We now have two equations and two unknowns we can solve.

           Hs (on):   34.0' = 2*SD + IC
           Hs (off):  30.8' = 2*SD - IC

Subtract the equations to get: (34.0' - 30.8') = 2*IC, or IC = 3.2/2 = +1.6, which is on the scale. Then add the equations to get: (34.0' + 30.8') = 4*SD, or SD = 64.8/4 = 16.2'. This arithmetic is worked in the form below.

That is the principle of the measurement that leads to the design of our form for the process. The best practice, as recommended in 1766, is to do this several times and then average the results.

In our proposed method, we add one more level of organization to the measurement, which comes from our extensive work with plastic sextants. That work has shown us that the IC you measure, depends on the direction you approach the alignment from. This is a much bigger effect in plastic sextants than in a good metal sextant, but even the best metal sextant will show this effect to some degree. In plastic sextants such as the Mark 15, there is notable slack in the gears. There is much less of this in a metal sextant, but you can, for example, still often align the horizon and stop, and then make a very slight turn of the dial before you decide it is no longer as well aligned.  Or align it, then see how much you can turn the dial before you note it is unaligned.

This is not just a mechanical issue, it is also related to our human interpretation of alignment. This varies between our perception of two images coming apart compared to two images coming together.  Thus we recommend measuring IC in both directions. In Step 3 we rotated counterclockwise till they came apart and Step 4 we rotated clockwise until they came apart. That illustrates the principle, but not the way we recommend doing it. We want to turn the dial in the same direction for the on and off values of the alignment.  In Step 3 we want to overshoot the alignment when bringing the reflected image down, and then turn the other way to align at the bottom and then top of the direct view going the same direction.

Each navigator needs to develop a way to keep track of which way they are turning the dials. What I have found useful is to think of a counterclockwise rotation to the left as "away" from me, thinking of the numbers on the top of the dial rotating away from me as I hold the sextant when reading the dials. This rotation increases the dial reading as the reflected sun moves down the view in the telescope.

With the same reasoning, I think of the other direction as "toward" me. This is in the direction of smaller numbers, with the reflected view rising in the telescope. This is clearly personalized terminology, but it helps me remember when I am turning "away," Hs is getting bigger, and the reflected sun is moving down. Others could be happier with just CW and CCW.

A first attempt at the measurement makes it very clear we have to be organized. We are turning a dial right or left, but it is backwards to us as we turn it. The motion of the sun we see is the result of a double reflection so when the numbers get bigger the moving sun is getting lower, and we are trying to keep track of its position relative to the other sun, which in principle is not moving, but the location of both of them in our view is totally controlled by very slight motions of the sextant as a whole (pitch, yaw, and roll)—not to mention that the suns are essentially the same color, and what we see at any moment would look identical to us if we did in the other direction! Pause or lose concentration for any reason, and we are lost—and must start again.  Indeed, we have to know what it means to "start again."  It should be no surprise that we strive to impose some order on the process.

Figure 3. Sequence of sights in a solar IC measurement. The gray sun represents the reflected view that moves with the dial rotations, although viewed through the proper filter these suns have essentially the same color. 

In Figure 3 the sights start in position #1, which is what we see with the sextant set to near 0º 0' looking toward the sun (all filters in place.) We begin with Away rotations till the suns are well separated as in #4, then we start up with a toward rotation, slow and smooth till #7 and then read and record the Hs on value.  Then keep going up till #9 and record the Hs off value. These are then called the "toward" values.

Next we repeat as in the bottom picture to get the "away" values, and these are recorded in the form and repeated several times as shown below.

Figure 4. Starpath Form 109, for solar IC measurements, designed by Lanny Petijean. His data here are using an Astra IIIB sextant, which was subsequently used for multiple position fixes on land to within 0.4 nmi as well as lunar distances accurate to within 30s. 

We see from these measurements the range of variation that is observed, even in the hands of an experienced user.  I must stress again that this method is not part of standard cel nav; it is a way to enhance the accuracy of sextant sights some fraction of an arc minute. I did two complete ocean passages in the 1980s by cel nav exclusively, and had never heard of this method at the time. Other factors such as good DR and logbook records are much more important for ocean navigation. On the other hand, attempts at accurate lunar distance measurements on land, should start with this method to measure the IC.

The sights are easier when the sun is lower, because we are looking more ahead than up, and the brightness diminishes with height as well, but there is a practical lower limit. An important part of this method is the check we get by measuring the SD of the sun, because we can look that up in the Nautical Almanac.  If they agree we have more confidence in our data.  In that comparison we make a tacit assumption that the refraction correction is the same for light coming from the top and bottom of the sun, which is perfectly valid for higher sights.

Refraction, however, changes rapidly for low sun or star heights. At Hs = 5º 0'.0 the correction is -9.9' but at Hs = 5º 32.0' (the other side of the sun) the correction is -9.1'. In other words, if we did our IC measurement with the sun 5º above the horizon, even with perfect sights we would this discrepancy of 0.8' confusing our analysis in some complex way. This discrepancy drops to 0.2' at Hs = 10º and is gone above 20º, so the practical lower limit  on this method to avoid that confusion is about 20º, which is about a handwidth above the horizon.

Sextant Sun Shades and Custom Filters

All sextants have two sets of shades, the main ones used for all sun sights called the index shades,  that cover the index mirror, and another set of usually much thinner shades called the horizon shades that cover the horizon glass, in direct line with the telescope.  These are intended for removing glare on the water, which is often an issue to deal with.  These horizon shades, however, are usually never thick enough to protect against a direct view of the sun as required in this measurement. I have seen sextants with a thick horizon shade likely intended for this purpose, but this is rare. Also we have a sextant with a pair of cross-polarized shades for both index and horizon (Figure 5).  These can in principle be set dark enough for this application, however...

...after using very many sextants of all types and price ranges, I have never seen a sextant that totally blocks out a view past the shades along one of their edges. This means that a standard sextant without a custom sun filter or eyepiece is not likely to be safe for this measurement.
In some older sextants we find an eyepiece cap that is thick like a welder's glass. These are presumably intended for these solar IC sights. You cannot see anything through them but the sun, and they go on the eyepiece end of the scope so both suns get shaded this way, resulting in two suns of the same color. Sometimes a colored sextant shade can be inserted that will alter the tone or color of one of the suns, which helps when possible.

Figure 5. A Tozaki sextant with several features oriented toward the solar IC measurement. The sextant shades are crossed polarized films, which rotate for varied shading.

This sextant does include the welder's shade eyepiece that fits over the main telescope, but even with this, we need an extra screen around telescope so we cannot inadvertently glance at the sun.  It is not just during the sight times we need protection; we must also lift the sextant and point to the sun, and that step also needs to be protected.

The 10 power scope is intended for IC measurements using the horizon, but this one also includes a welder's eyepiece, which must imply some optimistic thought on using it for the solar IC as well. This measurement is hard enough with a 4x40 or 7x35 scope; it takes a stronger arm and more patience than I have to succeed with the 10-power scope with its very small field of view.

The procedure for making a "Baader solar filter" for the end of the scope is given below.  Figure 6 shows one way to make a simple extra screen to prevent looking around the corner of the scope into the sun as you set up the sights.

Figure 6. "It may be sometimes convenient to provide an umbrella of pasteboard, about six inches square, with a hole in the middle to receive the telescope, in order to defend the eye from the direct light of the sun, as well as from the ambient brightness of the sky, which would otherwise render this practice in many cases too painful and difficult." —Nevil Maskelyne, 1766

It could be that telescope or camera stores have ready made filters that can be used on either the eyepiece or object side of the telescope for this purpose, but we have found it easy to construct our own that custom fits the various scopes we have, which vary from 7x35 on down to a Davis 2x20.  The procedure was first presented in our book How to Use Plastic Sextants: With Application to Metal Sextants and a Review of Sextant Piloting.

How to Make a Baader Sun Filter

The filter is named after the museum that originated the solar filter film we used. Alert: these films are expensive. See also links on related films as well as our earlier note on using your eclipse viewing sunshades for this—which is actually another approach, maybe even best. Make a pair of these that will stay securely in place, and do all the sights that way—short of reading the dials!

First check your sextant-telescope geometry so you know how much room you have. Generally the filter tube assembly must be made fairly thin to allow the index arm to move past it without hitting it.

Step 1. Wrap several strips of thin cardboard around the telescope, to form a small tube about 1” tall that just fits on your telescope. Glue these layers together to make the tube.

This shows the final filter, including how it started by wrapping layers around the telescope to get the right dimensions. We used our Smart Mark book marks for this, showing once again how smart they are. There is another finished filter shown in Figure 5.

Step 2. With the Baader foil between two thin cardboard or paper sheets mark the size of piece that will be needed to cover the tube and cut this out. Here we used card stock that was originally the cover of a booklet on weather.

Step 3. Make two cardboard base plate rings that have inside and outside diameters just a few mm smaller and bigger than the diameter of your tube. The film will be secured between these.

Step 4. Glue the tube to the center of one of the rings. Below we show a pocket watch being used as a weight as this glue dries. We used Gorilla glue, that dries white in 15 min or so.

Step 4. Put a trimmed layer of double-sided adhesive tape on the top of the ring. This will be used to hold the foil on the end.

Step 5. Carefully place this adhesive side down onto the foil to stick it to the ring, then add the second ring on top of that using the same adhesive tape to protect the edges of the foil. Small wrinkles in the foil will not matter, but you can usually do this with very few wrinkles.

Step 6. Trim the edges of the rim as much as you can, and be sure that at least one orientation of the filter will allow the index arm to pass below it.

Step 7. Look for some fortuitous container that can serve for storage and protection. We found a plastic pill jar just right for this one, with a few pieces of foam inserted to hold it in place.

With one of these filters on the end of the telescope and one of the extra screens mounted around the body of the telescope you are protected during the process.

Original Tables Requisite Instructions

Here is a video illustration of how the suns look through a telescope with sun filter in place, along with a few other comments on the process.

The video failed to show what it looks like to rock the sextant, which is illustrated in the image below. The effect depends on the extent of side error, but we are assuming we remove side error for this measurement. Generally you can tell the vertical alignment of the suns rather easily. The main thing to catch is if they are not one on top of the other, then you need to rock (roll) the sextant to them in line.

Note that the alignment does change when they are not vertical.