Compare Pub 249 and Pub 229 Using the Celestial Tools App

Celestial navigation sight reduction when done manually these days is primarily with the aid of Pub 249 or Pub 229. The former was developed for aviation; it is somewhat easier to use than Pub 229, the marine navigation standard, but is less precise in that air navigation by cel nav is less accurate to begin with because of the speed of the aircraft.

Nevertheless, Pub 249 remains popular with many ocean sailors. All USCG exams, on the other hand, must be solved using Pub 229.

Pub 249 rounds calculated heights (Hc) to the nearest whole arc minute, making them effectively ± 0.5' in precision, with azimuths (Zn) given to the whole degree, meaning ± 0.5º, whereas Pub 229 gives the tabulated values of Hc to the nearest 0.1' and azimuths to nearest 0.1º.  This does not, however, tell us how much a position fix might vary using these two different solutions. There are too many variables involved in that we have to look at the intersections of plotted lines of position that can be in more or less random directions.

We can look into this difference using an extensive practice exercise on running fixes (Exercise 6.6) from our textbook Celestial Navigation, which we can solve using a unique function of the free PC software app called Celestial Tools written by the late Stan Klein, an innovative and dedicated navigation instructor. Stan originally developed this tool for Power Squadron (USPS) members and instructors, and then made it available to the public. We have hosted his latest versions of the app for many years, and continue to so so, as was his wish.

There are 5 more exercises like this one in the textbook, each yielding practice with 3 running fixes (1 to 2, 2 to 3, and 1 to 3).  They cover all hemispheres and various course headings.

This is the graphic index from Celestial Tools. The function we are using is "SR Methods & Fix," which is a tool that tells us each of the intermediate answers to a sight reduction process for the various tables available plus the direction computation.  There is a description of the  all of Celestial Tools functions that supplements the included Help file. Find all of these in the Starpath Downloads page.

This exercise offers three running fixes we can use to check the differences between using 229 and 249. It is a unique form of exercise, copied from USCG license exams, which I will come back to. All sextant corrections are done; all Nautical Almanac lookup is done, so we are left to using the sight reduction tables to find Hc and Zn and then plotting two or more of the LOPs and advancing them to the same time for a running fix.

Below are the the data screens from the SR Methods & Fix function using 249 and 229 for the 1015  sight.

The first step is updating the DR to the times of the sights starting from the given position at 0900 WT. These results are shown below.

1015 sight reduced with Pub 249

From Pub 249 we get Hc = 58º 53' with a = 17' T 099º (which rounds the measured Ho minutes of 9.7' to to 10' according to 249 instructions, but we have no real reason to do that). A better result is Pub 249:  a = 16.7' T 099.

A short digression about the bottom center output show: The USPS and the Coast Guard Auxiliary define what they call an estimated position as the nearest point on an LOP to the DR position. That accounts for what you see here in this output labeled EP. This concept, however, has no real practical value for ocean navigation under sail, which is our main student body. Hence we not only do not teach this, we strongly discourage its use. It is much more likely to be misleading than to be helpful.

The best way to determine an "estimated position" is to use the log and compass DR corrected for everything you know of that might shift that position. In our terminology, we do not distinguish a DR position from an estimated position. This is the historical definition of a DR position that we adhere to. Certainly the best EP will indeed be located someone on the latest LOP if you have one. That is a definition of the LOP, but the nearest point on the LOP to the corresponding DR is not inherently anything more than uninformed wishful thinking.

1015 sight reduced with Pub 229

From Pub 229 we get Hc = 58º 53.6' with Pub 229: a = 16.1' T 097.6º  (Stan chose to round this to 098, but we do not need to do that. With care, we can plot to fractions of a degree.)

This example finds the a-values differing by 0.6' and Zn differing by 1.4º between 249 and 229.

Now we look at the 1156 sight.

1156 sight reduced with Pub 249

From Pub 249 at 1156 we get Hc = 80º 16' with a = 22' T 139º.

1156 sight reduced with Pub 229

From Pub 229 at 1156 we get Hc = 80º 16.4' with a = 21.6' T 135.8º.

Now we carry out a running fix between 1015 and 1156 sights using both reduction methods. We will compute this plotting as we are looking for small differences, but it could be done by hand.

From Pub 249
for the 1015 sight we have: AP 22 N, 124º 47.1' W,  with a = 16.7' T 099º
for the 1156 sight we have: AP 22 N, 125º 4.1' W,  with a = 22' T 139º

With speed 13.0 kts on course 250, we have a run time of 1h 41m and a distance run of 21.9 nmi.

From Pub 229
for the 1015 sight we have: AP 22 N, 124º 47.1' W,  with a = 16.1' T 097.6º
for the 1156 sight we have: AP 22 N, 125º 4.1' W,  with a = 21.6' T 135.8º

From the computed intersections below, we find that the pub 249 fix at 1156 is  21º 39.6' N,  124º 53.2' W and the Pub 229 fix is at  21º 39.3' N,  124º 53.7' W.  

Note the special convention on angle formats in this calculator.

Rfix 1 to 2 using Pub 249,

Rfix plot 1 to 2 using Pub 229.

These two positions are just 0.55 nmi apart, which is quite a surprise in that the 1156 Zns differ so much (139 vs 135.8).

________________________

We can now do the third sight reduction at 1425 and get another rfix from 2 to 3.

1425 sight reduced with Pub 229

From Pub 229 we get Hc = 60º 42.6' with Pub 229: a = 12.3' T 261.1º  (Again, we do not need to round Z when figuring Zn.)

1425 sight reduced with Pub 249.

From Pub 249 we get Hc = 60º 42' with Pub 249: a = 12.9' T 259º 

This example finds the a-values differing again by 0.6' (a coincidence) and the Zn differing by 2.1º.  For the rfix from 2 to 3 at 1425 we have:

From Pub 249
for the 1156 sight we have: AP 22 N, 125º 4.1' W,  with a = 22' T 139º
for the 1425 sight we have: AP 22 N, 125º 18.1' W,  with a = 12.9' T 259º

From Pub 229
for the 1156 sight we have: AP 22 N, 125º 4.1' W,  with a = 21.6' T 135.8º
for the 1425 sight we have: AP 22 N, 125º 18.1' W,  with a = 12.3' T 261.1º

From the computed intersections below we get for the pub 249 fix at 1425 of  20º 47.4' N,  125º 17.0' W and for the Pub 229 fix: 20º 41.5' N,  125º 18.3' W  

 Rfix 2 to 3 using Pub 249

Rfix 2 to 3 using Pub 229

These two solutions differ by 6 nmi, which is quite different from the sight 1 to sight 2 offset between sight reduction tables. 

We begin to see the issue involved. In the first case the sight reduction differences were larger than in the second case, but the second case had 10 times larger offset in final positions. The first difference was frankly smaller than I would have guessed, and the second one was larger than expected. Keep in mind that the 229 solution is the better one, and we are seeing here the cost of the lower precision of 249. These differences are mostly due to the Zn values and we can improve on the 249 values with interpolation, as outlined below.

____________

We can do one final test with a running fix from sight 1 to sight 3, and for these we have the solutions already done.

From Pub 249
for the 1015 sight we have: AP 22 N, 124º 47.1' W,  with a = 16.7' T 099º
for the 1425 sight we have: AP 22 N, 125º 18.1' W,  with a = 12.9' T 259º

With speed 13.0 kts on course 250, we have a run time of 4h 10m and a distance run of 54.2 nmi.

From Pub 229
for the 1015 sight we have: AP 22 N, 124º 47.1' W,  with a = 16.1' T 097.6º
for the 1425 sight we have: AP 22 N, 125º 18.1' W,  with a = 12.3' T 261.1º

The pub 249 fix at 1425  is at  21º 29.3' N,  125º 25.8' W

The pub 229 fix at 1425 is at  21º 28.9' N,  125º 26.3' W  

We then compute the distance between these two:

And we see that they differ by just 0.6 nmi, which is similar to what we saw in the sight 1 to sight 2 rfix. In this case we can also compare the fixes we got from 1 to 3 with what we got from 2 to 3.

In principle the 2 to 3 would be the better fix in that it is a shorter run so the errors in the DR are less crucial, and also the 1 to 3 sights let the sun get too far west so the intersection angles are not as good: 2 to 3 intersection at 54.7º and 1 to 3 at 16.5º.

2 to 3
Pub 249 fix at 1425  is at 20º 47.4' N,  125º 17.0' W 
Pub 229 fix at 1425  is at 20º 41.5' N,  125º 18.3' W  

1 to 3
The pub 249 fix at 1425 is at  21º 29.3' N,  125º 25.8' W

The pub 229 fix at 1425 is at  21º 28.9' N,  125º 26.3' W  

It is left as an exercise for those who want to study this more to work through the remaining examples using this procedure.

One way to improve the values of Z for any tables we use is to interpolate Z for the declination minutes. The standard procedure with 249 or 229 always takes Z from the degrees part of the declination, which was 14º in these examples. In all cases, however, the dec minutes were very large, so we would have gotten better results by using 15º dec to find the Z value. In this example it would have got 97 for 249 and 97.4 for 229, making the final fixes a bit closer still.