4.3.1 The record linkage process
Back to top

The Validation Study relies heavily on record linkages between the NCSCH, the NDI and the CMR and the veterans’ and children’s records. Over 65% of validated responses have been derived through this process. It is therefore important to discuss the technique and its application and any impact it may have on the results.

The linkage process is a probabilistic one. This is because not every veteran or veteran’s child can be uniquely identified. Identification of individuals by name, date of birth, sex and some details about the condition (e.g. date of diagnosis or death) across separate databases brings with it some risk of incorrectly matching two individuals together (false positives), or not linking the same individual (false negative). This occurs as a result of incorrect details in the databases for the same individual (e.g. birth dates) and also when there is more than one individual with similar characteristics.

The automated record linkage system used in the Validation Study is Automatch. This system is able to allocate a weight to a matched pair which is an indicator of match strength—based on a comparison of names, transpositions of name components, phonetic and common variants of name components, dates of birth and minor variations of these dates. It also takes account of the frequency with which these characteristics occur in the population, e.g. Smith (common name) or Lexcen (uncommon). This weight and the matched pair are also subject to a clerical case-by-case review before being accepted as a correct match.

In undertaking the record linkage process, as much information as was available was used to ensure that the match was correct. Unfortunately, this was made more difficult because the data file for the veterans contained mostly initials for the given names, not full names. This increased the risk of false positive matches (e.g. John Smith and Joe Smith would be equivalent as J Smith). Also, the risk of false negative matches was increased as possible matches were discarded because not enough information was provided to accept the match. Some of this risk was offset through an automated or clerical review of other information concerning the veteran, e.g. data of birth or address information. However, the review process took a conservative approach, only accepting matches that were of good quality.

4.3.2 NCSCH and State and Territory cancer registries
Back to top

Data quality and limitations

Cancer registry data are collected under State and Territory legislation, and therefore have excellent coverage compared with other health data collections. Coverage in the 1990s is estimated to be between 95% and 98% of new cases. Legislation has been in effect since 1982 in most jurisdictions with registrations prior to this occurring on a voluntary basis. In the Australian Capital Territory the cancer registry only became operational from 1994. This may have the effect of under-counting cancers diagnosed in the Australian Capital Territory prior to 1994 and is known to have affected the registration of melanoma in the Australian Capital Territory. Also, melanoma was known to be under counted in New South Wales for the period 1972–1988, due to poor registration of this cancer through pathology laboratories. Data collection in the Northern Territory was also known to be poor in the 1980s across all cancer sites. To compensate for the lack of coverage, any cancers that were diagnosed before 1982 and after 1996 were also sent to a clinician in the Validation Study.

In order to test the sensitivity of the linkage process a set of 105 cancers that had been confirmed by clinicians (between 1982 and 1996) were also linked to the NCSCH—53% of these cancers were found on the NCSCH.

Of the cancers not on the NCSCH, 55 were reported by veterans and their clinicians as melanoma of the skin and cancers of the head and neck. In this study it has been shown that a substantial proportion of reported head and neck cancers are in fact cancers of the skin. For the purposes of this analysis these cancers will be treated as melanoma. These types of cancers have two specific problems that make their identification on the NCSCH problematic. It is possible that a proportion of these cancers may not have been found to be melanomas on histological examination, but non-melanocytic skin cancer, and therefore are not reported to the NCSCH. It is also possible that clinicians removed the skin lesions (melanoma or non-melanocytic skin cancer) without sending the excised tissue for histological examination. Therefore, a melanoma diagnosis would not be registered on the NCSCH. In both scenarios, the clinician’s notes may have indicated melanoma or suspected melanoma. These scenarios would account for some underestimation in the record linkage process. By removing the impact of the melanomas in the linkage assessment, then the concordance between the clinician validation and validation using the NCSCH increases to 62%.

Table 4.4: Quality assessment of the record linkage process

In appraising the impact of this concordance level on the validation results, three key issues need to be considered:

• whether the cancers were not found due to the record linkage process;
• whether the cancers were not found on the NCSCH as they were never registered;
• whether the cancers reported by the veteran/clinician were in the time period corresponding with the NCSCH.

In the sample of 105 cancer cases, 49 cancers were not found (Table 4.4). In an intensive assessment of these missed cancers, it was found that

• 8% failed as a result of the record linkage process (due mainly to a lack of complete date of birth information);
• 67% were never registered by the State and Territory cancer registries and therefore were not available to the NCSCH; and
• 25% were found to be registered in 1997 by the cancer registries, not 1996 as reported by the veterans/clinicians. Currently, 1997 data is not available on the NCSCH.

If the last two factors are removed from an assessment of the record linkage process alone then the concordance between the two validation sources improves substantially from 53% to 93%.

However, it is important to note that two thirds of the sample were not registered by the cancer registries, and therefore were not included in the generation of the community comparison. This is because the Morbidity Study community comparison was based on the NCSCH data. This situation eliminates the impact of these missed cancers for the purposes of significance testing. However, if for other purposes the exact number of cancer cases needs to be known, then the number of estimated validated cancer conditions in Tables 3.3 and 3.6 would need to be increased by up to 30%. This would need to be assessed on a site by site basis as the concordance rate shows some variation.

Similar techniques to those described above were used when matching deaths in veterans or their children to the NDI, but here more accuracy would be achieved on such a well-defined event.

4.3.3 Registrars of Births, Deaths and Marriages and the National Death Index
Back to top

Data quality and limitations

Deaths are registered in each State and Territory under legislation that has operated since the beginning of the century. Australia has a good death data collection system when compared internationally, particularly when a cause of death (such as suicide) is used as an index of quality.

All deaths occurring since 1980 have been electronically registered and transferred to the NDI. Areas of concern in death registration relate mainly to the data collection process that relies on information provided by the next of kin. Often this information can be inexact due to the emotional circumstances under which it is provided. This often translates into variations between official names and names by which the person is known, estimated birth dates, or missing or poorly transcribed information. Occasionally deaths may be missed due to the transfer problems between coronial inquests, the standard registration system and the NDI. The quality of information has also changed over time, with dates of birth being recorded in all jurisdictions after 1995. Prior to this date, only three jurisdictions recorded this information and other jurisdictions used age at death from which an approximated year of birth could be calculated.

Cause-of-death information depends on the information provided by the certifying clinician on the death certificate. Until 1997, only a single underlying cause of death had been used to report cause of death on the NDI, despite other related causes of death being recorded on the death certificate (multiple causes of death). The effect of this may be that some causes of death reported in the Morbidity Study by veterans may not be reflected in the cause-of-death data. This leads to an underestimation of the numbers of deaths allocated to a particular cause. The underestimation is somewhat offset by the breadth of the categories used in the Morbidity Study which tends to aggregate a range of deaths, e.g. all cardiovascular-related causes of death are included in the illness category. There is anecdotal evidence to suggest an under-reporting of suicide and other causes of death (e.g. accidental drug overdose, HIV/AIDS) due to the social stigma and financial implications surrounding such a finding.

4.3.4 Clinicians
Back to top

Data quality and limitations

Clinical records held in institutions are usually active for only 7 years and after this some institutions destroy or archive records. This policy restricts the quantity of information about patients in research studies as institutions are loath to retrieve information older than this for non-clinical applications. This will have had an impact on the number of validations of conditions amongst veterans and their children, and particularly affect diagnoses prior to 1990.

Difficulties relating to clinical records are also encountered when dealing with general practitioners and specialists. Many of these clinicians will have retired, merged or sold their practice, or died. In many of these situations, patient records are often destroyed and follow-up is reliant on the current clinician having a complete patient history. This process also relies on the veteran or their child having provided the most appropriate clinician for the validation of their condition. In many instances, the clinician details provided by the veteran or their child were no longer appropriate and therefore no validation could be undertaken. No further clinician details were pursued with the veteran if their first option was unable to be contacted.

One advantage in pursuing medical records for this cohort was that the DVA and Central Army Records Office (CARO) also held medical history records. In other cohort follow-up studies, such comprehensive data collections under the control of key and sympathetic organisations are normally unavailable. These data sources proved important in validating older information about veterans, where CARO was the nominated validation source.

4.4 Non-respondents and ‘not able to be validated’ responses
Back to top