Are Spatiotemporal Methods Useful for Early Detection?

To answer the question in the title, let us start with the quotation from Fricker [1]: “Returning to the original question of whether statistical methods are useful for early event detection, I suggest that we really don’t know yet. That is, whether the systems and their associated detection algorithms can be modified so that they appropriately minimize false positive signals while maintaining sufficient sensitivity to actual outbreaks is still an open question”. It was also cited in our post “Biosurveillance in Crisis” of December 2011. This quotation is related to the purely temporal approach that is implemented in most current syndromic surveillance systems, by using statistical process control (SPC) methods. The temporal, univariate methodology is well-developed, widely used and technically much simpler than all existing spatiotemporal approaches. Actually, spatiotemporal  surveillance is a generalization of purely temporal one and as such it inherits all the challenges of the latter. In addition, spatiotemporal methods have both theoretical and practical challenges of their own. Therefore, whether statistical methods are useful for early event detection within spatiotemporal biosurveillance still is an open question even to the greater extent, than for temporal surveillance. Thus, as to early detection, spatiotemporal methods are unlikely to provide any advantages over temporal ones.

We have come to the above conclusion merely by comparison. The more important argumentation is as follows. In November 2011, the CDC overhauled its nationwide biosurveillance program, Biosense (see [2]). One of the most important components of this overhaul is giving more power and initiative to local jurisdictions - now  they have ownership of the data and the earliest checking of it.  “Local and state health departments have the best relationship with providers. They understand the context in which an event has happened, and they understand their population more than anybody else. So if we can make sure they have ownership of that data and the initial vetting of it is there, that would be the basis to truly start stitching a regional and national picture,” said Taha Kass-Hout, the CDC’s deputy director for information science and program manager for BioSense. Also Kass-Hout said: “BioSense will help the community ‘open for business’. That is, any health department in the country could ask their providers to share healthcare information with them in a meaningful ready to use environment. That will remove a lot of the barriers from the providers as well as the health departments.”

With this new data-sharing approach, in which local health departments – not the CDC – maintain ownership of their data in the form of daily counts of visits to local providers: emergency departments of city hospitals, affiliated clinics and doctor offices; and a new understanding of who is responsible for early analysis of this data and eventually for early decision making and response; it becomes clear that this new Biosense environment is ideal for purely temporal approaches applied at a very local, city level.

Now, let us briefly describe how a typical and one of the most commonly used spatiotemporal method, SaTScan works:

1.      A global region designated for surveillance (it could be the whole country or just a large geographical region, etc.) is subdivided into sub-regions.

2.      For each sub-region and a specified syndrome, the data are collected, typically in the form of visit counts during some baseline period comprising some most recent days.

3.      Then SaTScan searches for statistically significant clusters by comparing these counts in a certain geographical area with its neighboring areas. The algorithm is based on computing a likelihood ratio-based scan statistic and is using randomization to obtain p values (see, for example [3])

4.      See more details about SaTScan, its practical problems such as performance evaluation, computational time, etc, in [3] - [5].

Without considering any technicality, it is easy to see that SaTScan is a global method by design and by implementation, and as such it is left beyond the redesigned Biosense program, with its emphasis on local data ownership, local early analysis and local decision making and response. As says Burkom in [6]:   “I entered a recent project anticipating an application of scan statistics, but in the course of requirements and data analysis and give-and-take among the lead epidemiologist, implementers, and developers, we adopted a solution based on Bonferroni-limited multiple adaptive control charts”.

Thus, it has to be acknowledged that SaTScan as a typical spatiotemporal biosurveillance method can hardly be useful for early outbreak detection. As to situational awareness, it has to be based on some ability to predict the future development of the outbreak, which in turn should be based on some theoretical, epidemiological model. Since there is no any epidemiological component in the SaTScan methodology, it is unlikely that this approach could be helpful for situational awareness either. Most probably, SaTScan can be more successful in the static situations or slowly developing processes such as geographical distribution of cancer, diabetes, liver diseases etc., and also in non-health related applications: history, astronomy, demography among others (for more details, see [7]).

References

[1] Fricker, R. D. (2011a). Some methodological issues in biosurveillance. Statistics in Medicine, [full text]

[2] Goth, G. (2011). A new age of biosurveillance is upon us. http://www.govhealthit.com/news/new-age-biosurveillance-upon-us?page=0,1

[3] Shmueli, G. and Burkom, H. S. (2010). Statistical challenges facing early outbreak detection in biosurveillance. Technometrics, 52(1), pp. 39-51.                                                                                                            

[4] Fricker, R. D. (2010). Biosurveillance: detecting, tracking, and mitigating the

effects of natural disease and bioterrorism

http://faculty.nps.edu/rdfricke/docs/Encyclopedia%20Article.pdf

[5] Fraker, S. E. (2007). Evaluation of Scan Methods Used in the Monitoring of Public Health Surveillance Data (Dissertation) http://scholar.lib.vt.edu/theses/available/etd-11092007-111843/unrestricted/SEF-EDT.pdf  

[6] Burkom, H. S. (2011). Comments on ‘Some methodological issues in biosurveillance’ Statistics.in Medicine. 2011,30 pp. 426—429                                    

 [7] SaTScan Bibliography (2011). http://www.satscan.org/references.html

Epidemiological Significance vs. Statistical Significance

In Fricker (2011a), the author asks whether statistical methods are useful for early event detection and his suggestion is that he really does not know yet. Why so? First of all, because of the sequential nature of early detection, such fundamental concepts as significance level, power, specificity, and sensitivity cannot be used directly, without nontrivial modification. They are useful only for a fixed sample (Fricker (2011b). Secondly, biosurveillance data are usually autocorrelated, and even if such autocorrelation can be removed via modeling, the signaling statistics for early detection methods that use historical data in a moving baseline, are still strongly autocorrelated. As a result, again it is difficult to interpret specificity and sensitivity. Our approach to early detection is fundamentally different from the conventional ones. The mainstream approaches are based on removing autocorrelation from time series of daily counts by using some ad hoc regression methods, and then applying Statistical Process Control (SPC) charts to the residuals in regressions. Note that SPC charts were originally designed to work with uncorrelated data. Actually, the mainstream biosurveillance community considers autocorrelation as a nuisance. On the contrary, in our   approach autocorrelation is a major player: our only key parameter is the first-order autocorrelation coefficient, which  is related in a very simple way to the major epidemiological parameters, such as infection and recovery rates, and basic reproduction ratio R₀ (see [3] and also our previous post “Epidemiological Surveillance: How It Works”).

.          Since statistical inference methods for AR(1), including  parameter estimating, confidence Interval constructing and hypothesis testing are well-developed and easily available, it would seem that they could successfully applied to the problem of early detection and early situational awareness, but it is not the case.

Note also that in mainstream approaches, early detection and situational awareness are to some extent disconnected from each other, they are considered absolutely separate problems. And even if we have detected outbreak, for situational awareness we have to start from scratch since we have no information about further development of the outbreak. In our approach, we estimate only one parameter, the first-order autoregression coefficient in AR(1) approximation of SIR model, and we are able not only to decide whether the outbreak has already started, but also to get preliminary estimates of what we need for effective response and consequence management. 

           To the criticism expressed in [1], [2] and [4] regarding usefulness of such fundamental statistical concepts as statistical significance, p-value, sensitivity, specificity, etc for early detection, we can add some skepticism of our own. It is shown in [3] that both confidence intervals and hypothesis testing at 0.05 or 0.10 significance level are impractical for early detection purposes if we work with a typical sample size (baseline) of 7 – 14 days. For example, a hypothetical influenza epidemic as strong as Spanish flu cannot be detected in 7 – 14 days at 0.05 or 0.10 significance level. It is not a surprise because statistical significance depends mostly on the sample size: in very large samples, even very small effects will be significant, whereas in very small samples very large effects still cannot be considered significant. See for instance data borrowed from Table 13 in a classical book of statistical tables [5] with some linear interpolation

     

Critical Values of Correlation Coefficient r

 for Rejecting the Null Hypothesis (r= 0)

at the .05 Level Given Sample Size n

                        ______________________________________________

                               n                                                                   r

                        ______________________________________________

               

                               5                                                                 0.878.

                               7                                                                 0.755 (interpolated)

                             10                                                                 0.632

                             15                                                                 0.538 (interpolated)

                             20                                                                 0.444

                              50                                                                 0.276               

                             .……………………………………………………….

                      10,000                                                                0.0196 

           According to a rule of thumb (see [6]), r = 0.5 is considered a large effect, but  still it cannot be distinguished from null hypothesis r = 0.0 with sample size n = 15 at significance level of 0.05 since critical level is 0.538. At the same time, a negligible correlation r = 0.02 is statistically significant with n = 10,000.

Thus, the early detection goal cannot be achieved with such a small sample size as 7 – 14 days at any acceptable significance level. Instead, we propose to use the concept of practical, epidemiological, significance. Actually, what really matters is estimating the magnitude of effects, not testing whether they are zero. In our case, the effect is assessed by the parameter R₀, the basic reproductive ratio for the SIR model, and related to R₀ the first-order autoregression coefficient in AR(1) approximation of the SIR model. In [3] it has been proposed  the following early detection-combined-early situational awareness strategy:

(1)   Every day we estimate the first-order autoregression coefficient based on the moving baseline (from 7-day to 14-day);

(2)   With a very simple relationship between the autoregression coefficient and R₀, we actually estimate R₀ (below we use the same notation for the parameter R₀ and its estimate);

(3)   Then we compare the latter estimate with the known critical values for seasonal influenza (1.5 ≤ R₀ ≤ 3.0) and for Spanish Flu pandemic (3.0 ≤ R₀ ≤ 4.0);

(4)   Even R₀ ≈ 1 is worth of some field investigations;

If R₀ ≥ 1.5 then it is epidemiologically reasonable to report our findings as a significant risk of the epidemic;

If R₀ ≥ 3.0 then it is epidemiologically reasonable to report a severe risk.     

(5)   Knowledge of R₀ provides us with preliminary estimates of the number of

      infected at the epidemic peak and the total number of infected over the

      course of the outbreak.

Our critical levels (thresholds) have a very clear epidemiological meaning as opposed to rather arbitrary thresholds in the mainstream biosurveillance.

References  

[1] Fricker, R. D. (2011a). Some methodological issues in biosurveillance. Statistics in Medicine, [full text]

[2] Fricker, R. D. (2011b). Rejoinder: Some methodological issues in biosurveillance. Statistics in Medicine, [full text]  

[3] Shtatland, E. and Shtatland, T. (2011). Statistical approach to biosurveillance in crisis: what is next. NESUG Proceedings, [full text]                                                                   

 [4] Shmueli, G. and Burkom, H. S. (2010). Statistical challenges facing early outbreak detection in biosurveillance. Technometrics, 52(1), pp. 39-51.                                                                                                             

[5] Pearson, E. S. and Hartley, H. O. (Eds.). (1962).  Biometrika tables for statisticians (2^nd ed.).  Cambridge, MA: Cambridge University Press.                       

 [6] Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.

Epidemiological Surveillance: How It Works

This post is more technical than previous ones: it contains equations. Those who don’t like equations can skip the formulas and read the text only. In [1] we have used a Susceptible-Infected-Recovered (SIR) model mentioned in our post “Fusion of Biosurveillance with Epidemiological Modeling” (December 11, 2011). Here we discuss the SIR model in more details. Its graphical description is simple:

 Mathematically, the SIR model is described by the following first-order nonlinear system of difference equations:

S(n+1) = S(n) – (β/N)S(n)I(n),

I(n+1) = I(n) + (β/N)S(n)I(n)  - δI(n),                             (1)    

R(n+1) = R(n) + δI(n),

where S(n), I(n) and R(n) represent the numbers of susceptible, infected and recovered individuals correspondingly on day n; N is the total population (assumed constant); β is the infection transmission rate and δ is the average rate of recovery from infection. Note that d = 1/δ  is the mean duration of infectivity (in days). Both rates β and δ are supposed to be constant. Unfortunately, variables S(n) and R(n) are not observed or measured systematically in the biosurveillance real-time context. Only I(n) can be estimated, though indirectly, through the assumption that the overall number of infected on each day can be approximated by the sum of the number of patient visits to a hospital emergency department or a clinic or a physician office during the past d days (d is an infectivity period in days.) Therefore, we will work only with the second equation in (1). At the very beginning of an emerging disease or a new pandemic, which is the most interesting moment for early detection and early situational awareness, we can assume that there is no immunity in the population to the new disease, i.e., S(0) ≈ N. Hence, at the early phase of the epidemic, the second equation in (1) is reduced to the closed linear equation which contains only number of infected  
I(n+1) ≈ I(n) + (β – δ)I(n),              (2)       
Equation (2) is approximate (instead of = we have ≈), and taking into account errors of approximations we arrive at the stochastic equation
I(n+1)= I(n)(1 + (β – δ)) + w(n),              (3)                                                                    

where w(n) is an error or noise term (in more details see [1]). Formula (3) is the well-known equation of a first-order autoregression process AR(1). Equations (2) and (3) describe exponential growth if β – δ > 0 and exponential decay if β – δ < 0. Thus, the difference β – δ is a threshold parameter alternative to R₀ mentioned in our previous post “Fusion of Biosurveillance with Epidemiological Modeling” (December 11, 2011). Here R₀ can be expressed as R₀ =β/δ. Obviously, the threshold parameters R₀ and β - δ are equivalent:  

(R₀ > 1) <=> (β – δ) > 0 <=> Epidemic           

(R₀ < 1) <=> (β – δ) < 0 <=> Non-Epidemic

 In the early detection context, the advantage of the threshold parameter β – δ over R₀ is obvious: we have a linear parameter in linear equations (2) and (3) as opposed to nonlinear parameter R₀ =β/δ in a nonlinear setting of system (1). Our approach to computing R₀ is as follows: first, estimating the initial growth rate β – δ  through autoregression model (3) and then evaluating parameter R₀ through formula R₀   = 1 + 7(β – δ) (let us remind that in case of influenza we assume that the mean duration of infectivity d equal to 7 days). Thus, by making statistical inferences  about the only parameter β – δ in AR(1) model (3) we actually make inference about R₀. As a result, we are able to decide whether or not the epidemic has started (early detection task). If the answer is “yes” , we can  get preliminary estimates of the number of infected at the epidemic peak, the total number of infected over the course of the outbreak, the critical vaccination threshold,  etc. needed to develop measures for timely response and consequence management (early situational awareness task). See [1] and previous post “Fusion of Biosurveillance with Epidemiological Modeling” (December 11, 2011). Note that statistical inference methods for AR(1), including  parameter estimating, confidence Intervals constructing and hypothesis testing are well-developed and easily available (see [1]). Though, the situation is not as rosy as it looks. What exactly can be used from this statistical toolkit will be discussed in our next post.

And final remark, our approach could be called “Epidemiological Surveillance” (which is used in the title of the post). Really, on one hand, the approach is based on syndromic surveillance data (numbers of visits); on the other hand, the approach uses epidemiological models and their approximations for analysis and decision making. The problem is that this term (epidemiological surveillance) has been already used with the meaning of biosurveillance related to human health only (animals excluded!) (see [2]). We believe that our understanding of the term “epidemiological surveillance” is more to the point.

References

[1] Shtatland, E. and Shtatland, T. (2011). Statistical approach to biosurveillance in crisis: what is next. NESUG Proceedings. [full text]                                           

[2] Fricker, R. D. (2011a). Some methodological issues in biosurveillance. Statistics in Medicine. [full text]

Biosurveillance for Influenza

Influenza is the most widespread naturally occurring infectious disease in recent human history that has caused countless deaths worldwide. There were three influenza pandemics in the 20th century – the “Spanish” flu of 1918-19 (“the mother of all pandemics”), the “Asian” flu of 1957-58, and the “Hong Kong” flu of 1968-69. The 1918 flu, caused by a strain of H1N1, was by far the most deadly. Between 50 and 100 million people died totally as a result of the Spanish flu, possibly more than during the entire course of The Black Death. It makes the Spanish flu the deadliest natural disaster in human history. For comparison, death toll of WWI is estimated between 10 and 20 million, and WWII – between 62 and 78 million. The 1957 pandemic was due to a new H2N2 strain of influenza virus and killed two million people, while the 1968 pandemic resulted from an H3N2 strain and killed one million. In addition, there were 3 so-called flu pandemic scares (unrealized pandemics). The first pandemic of 21^st century was the Swine flu (April 2009 – July 2010). Fortunately, this H1N1 Swine flu pandemic was mild. It is convenient to show these results in the form of a table:

Pandemic            Years      Strain          Death Toll

Spanish flu      1918 - 1919    H1N1     50 - 100 million
Asian flu        1957 - 1958    H2N2            2 million

Hong Kong flu    1968 - 1969    H3N2            1 million

Swine flu        2009 - 2010    H1N1           25,174 (!)     

The names of influenza strains: H1N1, H2N2, H3N2, etc., are related to the two proteins that play a very important role in the process of influenza transmission: hemagglutinin  (where we get the H) and neuraminidase (from which the N comes from). It is known that there exist 16 different H-proteins: H1 – H16, and 9 different N-proteins: N1 – N9, overall 16 x 9 = 144 combinations. Although the most recent pandemic was rather mild, the uncertainty of “what is next?” is really alarming, Actually, pandemics happen every few decades. They occur when a new subtype of influenza A arises that has either never circulated in the human population or has not circulated for a very long time (so that most people do not have immunity against the virus), and it can spread easily through the human population.

Summarizing one can conclude that influenza is the number one permanent threat to the human population health, which deserves special attention and a particular approach. That is why in all developed countries there exist continuously operating influenza-specific reporting systems (such as FluView (CDC), EuroFlu (the WHO European Region), GoogleFlu (worldwide), FluWatch (Canada) etc. And that is why every day at 10 a.m. during flu season, a report on the number of cases of flu-like illness in the United Kingdom is placed on the desk of the British Prime Minister.

Fusion of Biosurveillance with Epidemiological Modeling

Usually, classical epidemiology is considered to be largely retrospective while biosurveillance should be prospective by definition. Recently, there have been published a few papers which can be viewed as a first step to the so-called real time epidemiology with the possibility of predicting further development of the epidemics (see the corresponding references in [1]). Still, those papers use weekly reporting, which is typical for epidemiology of infection diseases as a whole. In contrast, modern syndromic surveillance operates with daily counts (typically of emergency departments / clinics / doctor offices visits), and relying on weekly data is considered obsolete. And yet, the most important difference between epidemiological approach and syndromic surveillance is in information quality of the data used. Epidemiology uses weekly counts of cases with confirmed diagnosis whereas syndromic surveillance works with pre-diagnostic data. At the first glance, it is unclear whether these two types of data can be connected to each other, and consequently, whether epidemiological modeling can be of any use in syndromic surveillance. If the answer is ‘yes’, then we can use all the wealth of epidemiologic analytics for early detection and situational awareness goals. Below we will see the benefits syndromic surveillance can potentially get from its fusion with epidemiological modeling. The simplest and the most popular epidemiological model is the so-called susceptible-Infectious-recovered (SIR) model. SIR model is one of the early triumphs of mathematical epidemiology (since 1927!), and still it is a workhorse of modern epidemiology. We have also used SIR in [1]. There is a fundamental parameter in SIR, so-called basic reproductive number, R₀, which has a very simple epidemiological interpretation and can answer many questions regarding both early detection and situational awareness. First of all, R₀ is a threshold parameter, determining whether or not there is an epidemic: if R₀ is greater than 1 then the epidemic has started, otherwise there is no epidemic. Thus, R₀ can be used as a natural litmus test for early detection. Secondly, in terms of R₀  one can determine such important characteristics of the epidemic as: (1) the initial rate of increase of the epidemic during its exponential growth phase; (2) the number of infected at the epidemic peak; (3) the total number of infected over the course of the outbreak; (4) the critical vaccination threshold (=1/R₀.) etc. All these characteristics are very important for situational awareness. Therefore, using basic reproductive number R₀ allows us to timely detect an outbreak and simultaneously to evaluate the parameters of the rising epidemic in order to develop measures for timely response and consequence management. See technical details in [1]. However, all these results can be used in syndromic surveillance only if we can connect two different types of data: daily numbers of people with confirmed diagnoses and daily pre-diagnostic counts. Fortunately, it is possible at least in some very important cases (influenza epidemics and pandemics among them). It can be done thru the assumption that the overall number of infected on each day can be approximated by the sum of the number of visits to the emergency department (clinic or doctor office) during the past d days where d is an average number of days of infectivity. This is a reasonable approximation if parameter d is adequately estimated or chosen. In [1] (see also references therein) it has been supposed that for influenza, the mean duration of infectivity d is equal to 7 days. Indeed, most epidemiologists agree that influenza infectivity begins the day before illness onset and can persist for up to 7 days, although some persons may shed virus for longer periods, particularly young children and severely immuno-compromised patients. This assumption is clinically realistic and plays a fundamental role in our approach. It transforms visit counts, which are basic observable variables in syndromic surveillance, into the main epidemiological SIR variable – the number of infected. It is precisely the point where integration of syndromic surveillance with epidemiological predictive modeling is taking place. In this fusion, syndromic surveillance is an information provider (in terms of daily visit counts) and real-time epidemiology contributes to analysis and decision-making. As an additional bonus, 7-day summation allows to compensate for the day-of-week (DOW) effect in number of visits variability. Note that the DOW effect is a primary systematic feature of the data in all recent biosurveillance systems, which drastically influences their performance. The simple 7-day summation procedure is very effective in removing weekly patterns.

Thus, the above mentioned integration of syndromic surveillance and epidemiology can be effectively done in the important case of influenza.

But why influenza is so important?

References

[1] Shtatland, E. and Shtatland, T. (2011). Statistical approach to biosurveillance in crisis: what is next. NESUG Proceedings. [full text]

Biosurveillance in Crisis

Motivated by the threat of bioterrorism, biosurveillance / syndromic surveillance systems are now in crisis: with the original purpose of early detection, and more than 10 years in existence, no health department has reported using them for this purpose. There seems to be almost a consensus in the biosurveillance community that syndromic surveillance systems, based on statistical algorithms, are likely of little value in early detection of bioterrorist outbreaks; but irrespective of whether biosurveillance systems could be useful for detecting bioterrorism or not, their most important contribution to public health practice is detecting and responding to natural disease outbreaks, such as seasonal and especially pandemic flu (see Fricker (2011a and 2011b)).This has led to a shift away from only early detection of bioterrorist attacks. The goal has been expanded in two directions: firstly, to switch the emphasis from bioterrorism only to detecting and responding to natural disease outbreaks; and secondly, to include both early event detection and situational awareness, so that the focus is not simply on detection, but also on timely response and consequence management including vaccination and hospitalization strategies. Even with this expansion, early detection capacity is problematic. There are several reasons for that. One of them is uncontrolled alert rates: there is an alarm nearly every day and most health monitors learned to ignore alarms. It results in distrust in statistical methods and in biosurveillance itself. That is why Fricker, the most outspoken critic of the current state of biosurveillance, concludes in his recent review article [1] in such a pessimistic way: “Returning to the original question of whether statistical methods are useful for early event detection, I suggest that we really don’t know yet. That is, whether the systems and their associated detection algorithms can be modified so that they appropriately minimize false positive signals while maintaining sufficient sensitivity to actual outbreaks is still an open question”. Thus, not only early detection of potentially unpredictable bioterrorist attacks is considered “mission impossible”, but also early detection in general, including natural disease outbreaks. In [2], the rejoinder to the discussion of article [1], Fricker wrote: “And, in spite of my research interest in EED (early event detection) methods, I would suggest that situational awareness is probably the more important function for biosurveillance, since it enhances public health surveillance and management before, during, and after an outbreak“.

In our paper [3] it is shown that the situation is not as bleak and that it is possible, at least in some cases, for example in the case of influenza pandemic, to develop an approach which allows to achieve simultaneously both goals: early detection and early situational awareness, which is a unique ability in biosurveillance. No existing biosurveillance systems are capable of doing this. Such unification of early detection and situational awareness can be possible only through fusion of syndromic surveillance with epidemiological predictive modeling. See more on integration in my next post.

References

[1] Fricker, R. D. (2011a). Some methodological issues in biosurveillance. Statistics in Medicine, [full text]

[2] Fricker, R. D. (2011b). Rejoinder: Some methodological issues in biosurveillance. Statistics in Medicine, [full text]  

[3] Shtatland, E. and Shtatland, T. (2011). Statistical approach to biosurveillance in crisis: what is next. NESUG Proceedings, [full text]