Interpreting the Import Run Statistics

After each rail profile run is imported, a quantity of information concerning the processing is displayed in the Log Window and echoed to the daily log file. This section discusses the information piece by piece using examples.

You can control the detail level of this information by selecting "Minimal", "Standard", "Detailed", or "Full" from the Read Run or Import Run dialog. This section shows the Standard level of reporting unless otherwise indicated.

Run Statistics: Run Information

Basic run information is displayed when processing starts. The first line contains the run number and name assigned by the Office System, as well as the number and name known to the Field System.

Run 47 I_47 (in field system 23 CRO2_3645_980618).
Requested odometer range: -1e+006 to 1e+006.
Subdivision: 1 CALEDONIA.
Track: 2.
Run direction: S.

 

In the Office System, run numbers are assigned sequentially. The run name will usually be the same as in the Field System, unless it is the same as a run already in the Office System database. In that case a new name is assigned, as in the above example. The run name can be changed through the Run Table Maintenance dialog. Note: All file names, run names and comments must be in English. Names in other languages will generate an error message, preventing access to the affected databases. Also, the characters ; and { cannot be used in file or folder names.

Run(s) were imported successfully.

 

When run processing is complete, a status line indicates success or failure.

Run Statistics: User Options

The next section shows the modeling options that were in effect when the run was imported.

Modeling Options:

Segment Options:
Flyer Removal: Enabled
Minimum Gap: 2.5 mm.
Minimum Length: 3 mm.
Flyer Length: 10 mm.
Flyer Angle: 60 deg.
Head-web Gap: 25 mm.
Head Thinning: 0.0254 mm.
Base Thinning: 0.1 mm.
Head Modeling Options:
Center Gap: 20 mm.
Calibration Gap: 1 mm.
Head-only Top: 150 mm.
Head-only Width: 64 mm.
Head-only Height: 30 mm.
Rail Alignment Options:
One-side Alignment: Disabled
Head-only Models: Disabled
Max. Fitting Error: 1
Minimum Points: 60
Rail Selection Options:
Head Over Height: 1 mm.
Head Under Height: 15 mm.
Under Both Sides: 2 mm.
Face Angle Diff: 1.5 deg.
Camera Angle: 43 deg.
Web Thickness Over: 1.5 mm.
Web Thickness Under: 1 mm.
Fillet Radius Over: 2 mm.
Fillet Radius Under: 1 mm.
Use Imported Type: 50 %
Gauge Point Options:
Reference Distance: 0.625 in.
Maximum X Gap: 1 mm.
Maximum Y Gap: 4 mm.
Gauge Face Angle Options:
Minimum Wear: 2 mm.
Reference Distance: 0.625 in.
Below Reference: 7 mm.
Above Reference: 2 mm.
Minimum Coverage: 75 %

 

This is a useful reference when trying to fine-tune the option settings in order to improve the success rate of modeling rail profiles or of disambiguating similar types. See Modeling Options for more on how the modeling options work.

Run Statistics: Modeling Statistics

The Modeling Statistics section tells how successful the processing was, and reasons for failure:

Modeling Statistics (Standard):

Rail Profiles:
Raw Profiles: 13026
Full models: 10813 (83.0%)
Failed to model: 2213 (17.0%)
Bad head: 46 ( 0.4%)
Calib. gap too large: 13 ( 0.1%)
Not identified: 2167 (16.6%)
Could not fit: 32 ( 0.2%)
Too few near points: 1238 ( 9.5%)
Fitting error too high: 223 ( 1.7%)
No points on base: 1936 (14.9%)
Failed rail selection: 40 ( 0.3%)

Left rail calibration:
Average calibration gap: 0.11502 mm.
Gap too large: 8
Right rail calibration:
Average calibration gap: 0.06324 mm.
Gap too large: 5

 

It describes how the modeling options interacted with the data during processing. The statistics are your guide when fine-tuning modeling options for best performance.

Raw Profiles is the number of profiles processed (not necessarily modeled or saved) from the source data file. Other numbers in this section are expressed as a percentage of the number of raw profiles.

Full models are profiles that were fully modeled, including being identified as one of the selected rail weights. If One-Side Alignment is enabled, this will be followed by a line reporting the number of full models that were aligned using one side only. If Head-Only Models is enabled, the next line will report the number of 'head-only' models created during processing. These are not counted as full models, because no rail weight was determined.

The next section gives reasons profiles failed to model. Bad Head describes images that the software could not identify as a rail profile at all, or that have a serious defect that makes them unsuitable for measurement. One such defect is a 'calibration gap', which is too large. The calibration gap is the vertical gap, if any, in the portion of the head where the left and right half-profiles overlap. The Calibration Gap modeling option specifies a minimum acceptable gap. The example shows that 13 profiles were rejected as having a 'bad head' because the calibration gap was too wide. Information on the average calibration gap, and the number of profiles rejected on each side, is found in the next short section.

The 'bad head' statistics are immediately followed by statistics on profiles for which the rail type could not be identified, and reasons why. In the example, 2167 profiles (16.6% of the total) were not recognized as a specific rail weight. For 32 of them, the reason was "Could not fit": that is, the fitting algorithm could not fit those profile shapes to any of the selected rail types. The next 1238 were rejected because of 'too few near points', meaning that too few points (fewer than the 'Minimum Points' option setting) remained after the algorithm threw out 'outlier' points. (Outliers are points in the measured rail profile, which, after fitting to a selected rail type, are too far from the original unworn surface to be considered part of it. Outliers - which include measurements from worn surfaces of the rail head - are not used for alignment or rail identification.) Profiles with "too few near points" may be incomplete, or may have fit all the selected rail types so badly that many points were rejected as outliers.

Of the remaining profiles, 223 were rejected because the fitting error for all selected rail types was higher than the Maximum Fitting Error option. An additional 1936 profiles were rejected because no data points were found on the base of the rail (a condition usually caused by high ballast). Finally, 40 profiles were rejected because the results of the nine rail selection tests resulted in an unacceptable total error for all selected rail types. The rail selection tests are controlled by the Rail Selection options.

Run Statistics: Rail Selection

The nine rail selection tests help guide the software to select the correct rail type. The rail selection algorithm attempts to fit each measured rail profile to each of the selected unworn rail types in turn. If the fit to the unworn type is "good enough" (the fitting error does not exceed the Maximum Fitting Error setting), then the nine specific tests are applied. If the rail profile "fails" any of the tests, the "Total Error" is increased, and probability that the measured rail profile will be selected as that rail type is reduced.

The rail selection tests change the probability that a given rail type will be selected. We will use the same example where the measured fillet radius was 0.126 in. greater than the specified 136 RE fillet and the Fillet Radius Over option allowed 0.125 in. The new algorithm takes the difference (0.001 in.), multiplies it by a weighting factor, and adds the result to "Total Error". Since the amount by which the measured radius exceeds the specification is small, the total error is only increased by a small amount. The probability that the measured profile will be selected as 136 RE is only slightly reduced. If in other respects the measured profile looks more like 136 RE than any other rail type under consideration, it will probably be identified as 136 RE.

The new approach identifies rail sections more accurately. It also gives a higher overall yield of profiles successfully modeled. For details of how each rail selection test works, and how the total error calculation is affected by each one, refer to the Modeling Options section.

What follows is an example of rail selection statistics. There is a group of statistics for each of the unworn rail types selected during processing.

Rail Recognition Statistics:

Count Average Min Max Std Dev
RE115
Fit OK 9684 0.36693 0.08382 0.99590 0.13853
Over Height 0
Under Height 2
Under Both Sides 2
Face Angle Diff. 1
Camera Angle 0
Web Thickness Over 2
Web Thickness Under 13
Fillet Radius Over 29
Fillet Radius Under 7495
Changed to Imported Type 184
Different from Imported Type 9
Total Error - Pass 9659 0.39040 0.08382 0.99715 0.14153
Fitting Error - Selected 9657 0.36591 0.08382 0.99590 0.13677
Total Error - Selected 9657 0.39037 0.08382 0.99715 0.14152

Count Average Min Max Std Dev
JK124
Fit OK 1295 0.34424 0.08197 0.99994 0.18845
Over Height 1
Under Height 0
Under Both Sides 5
Face Angle Diff. 16
Camera Angle 0
Web Thickness Over 0
Web Thickness Under 1
Fillet Radius Over 3
Fillet Radius Under 174
Changed to Imported Type 0
Different from Imported Type 22
Total Error - Pass 1270 0.34710 0.08197 0.99994 0.18849
Fitting Error - Selected 1156 0.29968 0.08197 0.82941 0.12400
Total Error - Selected 1156 0.30492 0.08197 0.94864 0.12796

Count Average Min Max Std Dev
RE136
Fit OK 0
Over Height 0
Under Height 0
Under Both Sides 0
Face Angle Diff. 0
Camera Angle 0
Web Thickness Over 0
Web Thickness Under 0
Fillet Radius Over 0
Fillet Radius Under 0
Changed to Imported Type 0
Different from Imported Type 0
Total Error - Pass 0
Fitting Error - Selected 0
Total Error - Selected 0

 

For each rail type, the 'Fit OK' line includes all the profiles that were successfully fitted to the rail type with a lower fitting error than the maximum allowed. The left column shows the number of profiles, followed by the average fitting error, the minimum and maximum errors, and the standard deviation of the errors. In all the cases shown, the maximum error is just slightly less than the Maximum Fitting Error option. This is not surprising, because any profile with a higher fitting error is not counted.

The standard deviation can be helpful in judging the likely effect of changing the Maximum Fitting Error value. In a normal statistical distribution, 67% of the population lies within ± 1 standard deviation of the mean (average) value, 95% of the population lies within ± 2 standard deviations, and 99.7% within ± 3 standard deviations. Although the errors we are dealing with here are not distributed normally, you can use the standard deviation as a rule-of-thumb guide. For example, if 33% of profiles are rejected for the reason "Fitting error too high", you should be able to cut the rejection rate to about 5% by increasing "Maximum Fitting Error" by the amount of one standard deviation as reported on the "Fit OK" line.

The next nine lines after the "Fit OK" line record the number of profiles that exceeded the threshold for each of the nine rail selection tests. Looking at the statistics for 115 RE, we see that of 9684 profiles which fit the 115-lb profile with an acceptably low fitting error, 2 were classified as "Under Height", 2 (possibly the same ones) were classified as "Under Both Sides", and small numbers of profiles were picked out by other tests. The notable exception is "Fillet Radius Under", where 7495 profiles had a measured fillet radius that was less than the nominal 115 lb RE radius by an amount exceeding the Fillet Radius Under option. However, that did not prevent the software from identifying most of those profiles as 115 RE.

The next two lines are included if the Use Imported Type rail selection option is used. "Changed to Imported Type" reports the number of profiles that were changed to the type identified in the import source file. "Different from Imported Type" reports the number of profiles that were not changed to the imported rail type. The example for 115 RE shows that 184 profiles which Rangecam would have identified as a different, were changed to 115 RE because of the reduction in calculated fitting error specified by the Use Import Type option. (The reduction in this case is 50%). The next line shows that 9 profiles that were identified as a different rail weight in the source file, were selected as 115 RE despite the reduction in fitting error. If, in reviewing the results, you concluded that most of the 184 profiles changed to 115 RE were actually a different type, then you might lower the value of Use Import Type in order to increase the weight of Rangecam's selection. On the other hand, if it appeared that the rail identification in the import source file was usually correct, you might consider raising the value in order to increase the number of profile identifications determined externally rather than by Rangecam. Please see Modeling Options - Rail Selection for more on how the Use Import Type option works.

The last three lines of this section say that 9659 of the original 9684 profiles were an acceptable fit to the 115 RE rail type even after the rail selection tests were applied, and that all but two of those were actually selected as 115 RE. They were selected because their Total Error for 115 RE was lower than for any of the other rail types under consideration.

Statistics are given on the last two lines for Fitting Error and Total Error (Fitting Error plus adjustments for the rail selection tests) for the profiles selected as 115 RE.

If there are high numbers beside any of the rail selection tests, the test is having an effect. This means one of two things: either the test is helping to weed out profiles of a different rail type, or the rail selection option is set too tight, which may process. In the above example, all of the rail being measured was 115 RE or 124 JK. (No profiles were close enough to 136 RE for it to be considered.) The 115 RE and 124 JK rail sections happen to have the same base fillet radius, which is 0.75 in. (19.05 mm). So we know that the setting of 1 mm for the Fillet Radius Under option is more restrictive than it should be; 2 mm would be more appropriate.

The final section of the run statistics covers cases where the system could not derive measurements from the profile data.

Measurement Failures:
Could not find track gauge: 1778
Could not find gauge point: 4
Could not find head area: 0
Too little GFA coverage: 18

 

This says there were 1778 profile pairs where track gauge could not be measured, 4 individual profiles in which the gauge point could not be found, 0 profiles for which head area could not be calculated, and 18 profiles which had less than the specified amount of data on the gauge face to calculate gauge face angle.

You can change the success rate of measurement calculations by relaxing or tightening the Gauge Point and Gauge Face Angle options. For more information on measurement and modeling options, refer to the Modeling Options section.

Modeling Statistics: Alternative Levels of Detail

The Minimal detail level is only a few lines long. It is intended to provide a quick check on the system calibration and overall modeling success rate using an established set of options. It does not contain enough detail to guide you in tuning the options

Besides the standard statistics, the Detailed level includes information on profile pairs, half-profiles, segments, flyers, and the gap between head and web. It also has full statistics (not just counts) on the fitting errors of profiles rejected by the Rail Selection tests. Here is an example of what might appear after the "Failed Rail Selection" line:

Profile Pairs: 14089
Single Profiles: 646

Half-profiles:
With base: 57510
Without base: 332
Blank: 0
With flyers: 11872

Count Average Minimum Maximum Std Dev
Flyers:
Flyer Gap: 2.90937 2.50054 124.16084 1.77314
Flyer Length: 2.65291 0.00000 9.99987 4.08535
Head Gap: 39.56751 25.00000 115.20000 5.87219
Head Count: 110.62855 32.00000 318.00000 24.04231
Base Count: 122.32850 1.00000 218.00000 20.70648
Segments:
Segment Gap: 4.57130 2.50054 40.12014 3.67760
Short Segment: 23536 0.02641 0.00000 2.98718 0.25154
Head Segments: 114.09016 54.00000 224.00000 19.45035
Base Segments: 63.97174 20.00000 125.00000 13.98494
Left rail overlap:
Length: 9298 12.49608 0.00000 25.79999 4.99625
Calibration gap: 2 0.11308 0.00364 0.51744 0.05866
Gap too large: 2
No overlap: 4948
Right rail overlap:
Length: 8717 10.66499 0.00000 38.79999 2.96209
Calibration gap: 8710 0.09954 0.00667 0.63762 0.05552
Gap too large: 5
No overlap: 5861
Rail Center Gap:
No center gap: 26405
Center gap: 1157 17.52760 2.51996 40.12014 7.69179

 

Profile Pairs is the number of pairs of profiles at the same location on the left and right track sides. Single Profiles are profiles that are not included in a pair.

A half-profile is one side of a profile, as seen by one camera belonging to an optical measurement system. (Two cameras are required to see both sides of a rail.)

Half-profiles with base are ones in which the base and head portions could be distinguished. Half-profiles without base are ones where this distinction could not be made. When processing embedded track this number is normally high. If there are many half-profiles without base when processing exposed rail, it may be because the head-web gap option is set to too high a value.

Blank half-profiles are ones which contain no data points whatever. Many blanks may indicate malfunctioning hardware such as a weak or non-existent light-stripe, or other low-level problems such as desynchronization of the camera and the pulsed stripe of light.

Half-profiles with flyers are ones in which the software detected, and eliminated, flyers.

The next two lines contain statistics on flyers found. Flyer gap is the distance of the flyer from what is assumed to be the surface of the rail. Flyer length is the length of the flyer.

Head Gap is the vertical distance separating the head portion of a half-profile from the base portion, which begins with the web. Detection of the head gap is governed by the Head-Web Gap option. If the actual gap is less than the Head-Web Gap, the break between head and web will not be recognized, and the half-profile will be counted as Without Base. Here we see that the average gap is over 39 mm., the minimum is exactly 25 mm., and the maximum is 115.2 mm. The standard deviation is about 5.9 mm. In a normal distribution, 99.7% of samples fall within ± 3 sigma of the average. If we have reason to think that the 332 half-profiles 'without base' are in fact normal rail profiles, we may want to lower the value of the Head-Web Gap option so as to include them. The statistics can guide us in this process. We would try to find a setting for the Head-Web Gap option which would be slightly below the average gap minus 3 times the standard deviation: i.e. 39 mm. - (3 x 5.9 mm.) = 21.3 mm. However, changing the value of the option will change the average, minimum and sigma values, so finding the best number may be an iterative process.

Head Count and Base Count are the number of points found on the head and base portions of each half-profile.

The statistics on Segments have to do with lengths of continuous profile contour fragments, and the gaps between them. Segment Gap is the length of the gap between segments. Detection of a gap is controlled by the Minimum Gap option (set to 2.5 mm. in the example). Short Segments are contour fragments that are discarded because they are shorter than the Minimum Length setting. Head Segments and Base Segments are the number of segments used to model the head and base portions of each profile. These numbers directly determine the disk storage required for the profile shape. They are controlled by the Head and Base Thinning options.

Left Rail and Right Rail Overlap statistics replace the section on Left and Right Rail Calibration in the Standard and Minimal reports. More statistics are included: the length of the overlap area between left and right half-profiles, and the number of profiles where the halves did not overlap.

Rail Center Gap counts the profiles in which a gap was or was not found at the point where the rail centerline intersects the head. Vertical wear will not be calculated for profiles where the center gap is longer than the Center Gap option setting.

Besides everything in the Detailed level, the Full level includes information on the internal workings of the rail recognition and alignment algorithm: the average number of iterations required to reach convergence, the number of successes and failures. It is mainly of interest to developers of the software.