Survex drops Walls stations; possibly related to multiple levels of nested 'exported' shots

[Joe Walko]

See attached sample files - Running emily.wpj in both Walls and cavern will illustrate the issue.

This group of stations, all in the same part of the Walls tree, are the only stations being dropped in the large dataset. I've gone round and round on this, and can't quite pin it down, nor can I reproduce with simplified SRV examples.

Attached are four "as is" files from the dataset, along with a wpj and a 'fixed-point' SRV which reproduces the problem. Was not able to simplify it much beyond this level.

Note the very extensive "indirect" references to 'external' stations, some of which no longer reside in an SRV with a matching #PREFIX. This due to extensive resurvey and station-replacements, but they retained original station nomenclature when the old stations could be recovered.

This feels like an issue related to too much levels 'exported' nodes, but root causes eludes me.

I tested with Survex updates pulled earlier today, probably before c36617f08037a1c697fd40f4e98866f617e4a3e1 (related to REF and geo-location).

These log-file warnings seem relevant, but I can't discern why they happen: The following survey stations are not attached to a fixed point:
FSB3693.SRV:26: info: 2473:C132
FSB3689.SRV:47: info: 3626:H6
FSB3689.SRV:42: info: 3689:GB6
FSB3689.SRV:14: info: 3626:H2'

List of Walls stations missing from Survex:

3626:H6

3632:Z21

3689:A14

3689:H10

3689:H11

3689:H12

3689:H13

3689:H14

3689:H7

3689:H8

3689:H9

3690:H15

3690:H16

3690:H17

3690:H18

3690:H19

3690:H20

3690:H21

3690:H21A

3690:H21B

3690:R1

3690:R2

3690:R2A

3690:R2B

3690:R2C

3690:R2D

3690:R3

[Joe Walko]

Sample data and diagnostic info related to 'missing' stations in Suvex

[Olly Betts]

These log-file warnings seem relevant, but I can't discern why they happen:

The following survey stations are not attached to a fixed point: FSB3693.SRV:26: info: 2473:C132

I dug into this first one and it seems correct to me - grep -i '\<C1[34][0-9]\>' *.SRV seems to show that 2473:C132 is only connected to other stations in the range C133-C148 (with one of two prefixes), but none of those stations seem to be connected to a fixed point.

This station isn't listed in walls_emily_stations.LST (and neither are 3689:GB6 or 3626:H2') so it seems Walls also thinks these aren't connected to fixed points. (Walls doesn't seem to inform the user when there are dangling surveys, which seems a baffling design choice to me as failing to tie in a new survey is an easy mistake to make; connections can also be accidentally broken later on - e.g. when replacing a dubious section of survey with a resurvey.)

3626:H6 does have a position in Walls though, and at least some (perhaps all) of the missing stations you've listed look like they're in the same chunk of survey that Survex thinks is dangling. I'll have a quick go at reducing the testcase.

[Joe Walko]

With the full dataset, the issue was limited to stations with prefix 3690 listed below. The others in the list above are probably an artifact of my butchering the sample down to as few SRVs as possible, and those non-3690 stations can be ignored. I should have trimmed the others, before posting that list. (Note the Prefix2 of "M" here, since this was from the full/merged dataset, but that shouldn't be a factor).

M:3690:H15

M:3690:H16

M:3690:H17

M:3690:H18

M:3690:H19

M:3690:H20

M:3690:H21

M:3690:H21A

M:3690:H21B

M:3690:R1

M:3690:R2

M:3690:R2A

M:3690:R2B

M:3690:R2C

M:3690:R2D

[Olly Betts]

It looks like it's mishandling the nested books in the WPJ somehow and that results in it failing to actually process the last .SRV listed there, and that results in the missing connecting legs.

The logic for deciding where a survey item ends is a bit fiddly here. Perhaps we can assume an order the items appear in and simplify it, but without a format spec for WPJ I tried to assume as little as possible.

[Olly Betts]

It seems that .STATUS always appears as the final thing in an item, except that if the status is zero Walls seems to omit this line entirely. That is frustrating as otherwise we could simply process the item when we get to .STATUS.

[Olly Betts]

Oh! The missing .STATUS is the problem because I set the variable this goes into to -1 by default during development so I could see when it hadn't been set, but if it isn't present it seems that means the status is zero. If we default it to 0 instead your example works and checking with diffpos shows the stations all match and move by a few cm at most:

$ diffpos emily.3d walls_emily_stations.LST 0.02
Moved by (-0.03,-0.01,-0.01): 3693:H24

I've pushed a fix in cd07e2a6e7f922629de88cc5713ea7a07e1b192d. I'll add a testcase for it when I get a chance.

[Olly Betts]

5f4b4980e1014f99838507620dfb61b4b5665ad0 fixes a related bug and adds regression tests for both fixes.