1 Logistic & Poisson Regression: Overview

In this chapter, I’ve mashed together online datasets, tutorials, and my own modifications thereto. I start with the packages we will need. Then I move into data cleaning and assumptions. The model itself is possibly the easiest thing to run. Then, we wrap up with all the stats you’ll ever need for your logistic regression and how to graph it. Before we leave, we’ll look at the slight modification for running a Poisson regression.

1.1 Why would you do logistic regression?

Logistic regression, also known as logit regression, is what you use when your outcome variable (dependent variable) is dichotomous. These would refer to all your research yes/no questions: Did you vote, yes or no? Do you have a disease, yes or no? Did you use [insert your favorite illicit substance here] in the past week, yes or no? You might also think of it as pass/fail, hit/miss, success/failure, alive/dead, etc. Rather than estimate beta sizes, the logistic regression estimates the probability of getting one of your two outcomes (i.e., the probability of voting vs. not voting) given a predictor/independent variable(s). For our purposes, “hit” refers to your favored outcome and “miss” refers to your unfavored outcome.

1.2 Why would you do a Poisson regression?

Poisson regression, also known as a log-linear model, is what you use when your outcome variable is a count (i.e., numeric, but not quite so wide in range as a continuous variable.) Examples of count variables in research include how many heart attacks or strokes one’s had, how many days in the past month one’s used [insert your favorite illicit substance here], or, as in survival analysis, how many days from outbreak until infection. The Poisson distribution is unique in that its mean and its variance are equal. This is often due to zero inflation. Sometimes two processes may be at work: one that determines whether or not an event happens at all and another that determines how many times the event happens when it does. Using our count variables from above, this could be a sample that contains individuals with and without heart disease: those without heart disease cause a disproportionate amount of zeros in the data and those with heart disease trail off in a tail to the right with increasing amounts of heart attacks. This is why logistic and Poisson regressions go together in research: there is a dichotomous outcome inherent in a Poisson distribution. However, the “hits” in the logistic question can’t be understood without further conducting the Poisson regression.

1.2.1 Packages

We’re going to start by loading the following libraries. Here’s why:

  • aod: Stands for Analysis of Overdispersed Data, contains a bunch of functions for this type of analysis on counts or proportions. The one you’ll see the most in this chapter is wald.test (i.e., Wald Test for Model Coefficients.)

  • ggplot2: ggplot2 is an expansion on Leland Wilkinson’s The Grammar of Graphics. ggplot2 let’s you graphically represent both univariate and multivariate numerical and categorical data. Since we’re doing logistic regression, we need a graphing library that can handle categorical data.

  • effects: Another graphing package because options are good.

  • graphics: This package allows you to go beyond R graphing primitives. Below, you’ll see cdplot (Conditional Density Plots), xlim, ylim, and others in action.

  • car: Lets us modify simple plots in R with labels, titles, etc.

  • pscl: Need this to create a pseudo R-squared for logistic regression.

Note: You might get some Warning Messages when you load these depending on what version of RStudio you have and/or whether you’ve previously loaded these in the past. You may also need to go to into Packages>Install and check the box next to the package you want to load if you’re using RStudio. Usually, once you install a package, library() should be sufficient to load it in the future.

## 
## Attaching package: 'aod'
## The following object is masked from 'package:survival':
## 
##     rats
## Classes and Methods for R developed in the
## Political Science Computational Laboratory
## Department of Political Science
## Stanford University
## Simon Jackman
## hurdle and zeroinfl functions by Achim Zeileis

1.3 Now let’s load our data.

I’ll be bringing in a couple datasets freely available online in order to demonstrate what needs to happen in logistic regression. I basically string together things available in several places online so that we have everything we need for logistic regression analysis here in one chapter. See endnotes for links and references. We have 2 datasets we’ll be working with for logistic regression and 1 for poisson. We start with the logistic ones. Because we will be using multiple datasets and switching between them, I will use attach and detach to tell R which dataset each block of code refers to. Attach is a function that brings the dataset into action and detach phases it out again.

cuse <- read.table("http://data.princeton.edu/wws509/datasets/cuse.dat", header = TRUE)
mydata <- read.csv("http://www.ats.ucla.edu/stat/data/binary.csv")
#Notice we're not setting a working directory because we're reading in data from the internet.

1.4 Data Cleaning & Reading Output

…Because none of this matters if your data is not in binary form!

Let’s start by learning some ways to manage our data so we can use it for a dichotomous outcome.

The cuse dataset is an old one that includes an N = 16 women, age group in which they belong, their education level, whether they want more children, and whether or not they’re using contraceptives. Location is listed above. I use the dataset from Princeton and modify/add to the tutorial by G. Rodriguez. Here’s a look at the data to get a sense of it.

attach(cuse)
str(cuse) 
## 'data.frame':    16 obs. of  5 variables:
##  $ age      : Factor w/ 4 levels "<25","25-29",..: 1 1 1 1 2 2 2 2 3 3 ...
##  $ education: Factor w/ 2 levels "high","low": 2 2 1 1 2 2 1 1 2 2 ...
##  $ wantsMore: Factor w/ 2 levels "no","yes": 2 1 2 1 2 1 2 1 2 1 ...
##  $ notUsing : int  53 10 212 50 60 19 155 65 112 77 ...
##  $ using    : int  6 4 52 10 14 10 54 27 33 80 ...
levels(cuse$age)
## [1] "<25"   "25-29" "30-39" "40-49"
detach(cuse)

Here’s your basic glm model wherein the wantsMorefit (as in, they want more children: yes or no) object is our first model to fit. We specify the glm function, our outcome variable (i.e., wantsMore) and our predictors (i.e., age and education, without any interaction factors yet, hence we only use + for the main effects), we tell it what data to use (i.e., cuse), and that the family function for our distribution (i.e., binomial.) This model is asking, “Do age and education predict wanting more children? If yes, to what extent?” Here, our outcome (wanting more children/wantsMore) is already in binary form:

attach(cuse)
wantsMorefit <- glm(wantsMore~age+education,data=cuse,family=binomial()) 
summary(wantsMorefit) # display results
## 
## Call:
## glm(formula = wantsMore ~ age + education, family = binomial(), 
##     data = cuse)
## 
## Deviance Residuals: 
##    Min      1Q  Median      3Q     Max  
## -1.177  -1.177   0.000   1.177   1.177  
## 
## Coefficients:
##                Estimate Std. Error z value Pr(>|z|)
## (Intercept)   6.267e-16  1.118e+00       0        1
## age25-29     -4.047e-16  1.414e+00       0        1
## age30-39     -5.860e-16  1.414e+00       0        1
## age40-49     -6.280e-16  1.414e+00       0        1
## educationlow -4.441e-16  1.000e+00       0        1
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 22.181  on 15  degrees of freedom
## Residual deviance: 22.181  on 11  degrees of freedom
## AIC: 32.181
## 
## Number of Fisher Scoring iterations: 2

Some notes on the stats we generated above: Unlike linear regression, we’re using glm and our family is binomial. In logistic regression, we are no longer speaking in terms of beta sizes. The logistic function is S-shaped and constricts the range to 0-1. Thus, we are instead calculating the odds of getting a 0 vs. 1 outcome.

Let’s walk through the output: The first thing you see is the deviance residuals, which is a measure of model fit (higher is worse.) Next is our first batch of coefficients. Going from left to right, the Coefficients table shows you the “Estimate” change in the log odds of the outcome variable per one unit increase in the predictor variable, standard error, the z-statistic AKA the Wald z-statistic, and the p-value.

Here’s an example of how you’d interpret the coefficients above: For each unit change in [insert predictor variable], the log odds of [achieving the outcome of interest] increases by [coefficient].

You’ve never seen someone report anything in terms of log odds and you’re not sure how this is a meaningful description of the outcome variable, you say? That’s why we exponentiate: to put this in terms of odds ratios (see below for interpretation of ORs.)

confint(wantsMorefit) # 95% CI for the coefficients
## Waiting for profiling to be done...
##                  2.5 %   97.5 %
## (Intercept)  -2.343134 2.343134
## age25-29     -2.884060 2.884060
## age30-39     -2.884060 2.884060
## age40-49     -2.884060 2.884060
## educationlow -1.999555 1.999555

The next part of our output is the 95% confidence intervals (CI) for the unexponentiated-coefficients. In logistic regression, if the confidence interval crosses over zero, as in the interval stretches from a negative value to a positive value, that effect is not significant.

Here’s the same thing below, but this time with exponentiated coefficients:

exp(coef(wantsMorefit)) # exponentiated coefficients
##  (Intercept)     age25-29     age30-39     age40-49 educationlow 
##            1            1            1            1            1
exp(confint(wantsMorefit)) # 95% CI for exponentiated coefficients
## Waiting for profiling to be done...
##                   2.5 %    97.5 %
## (Intercept)  0.09602624 10.413821
## age25-29     0.05590729 17.886754
## age30-39     0.05590729 17.886754
## age40-49     0.05590729 17.886754
## educationlow 0.13539558  7.385765

Interpreting ORs: For each unit [“increase” if OR is positive/“decrease” if OR is negative], the odds of [achieving the outcome of interest vs. not] increases/decreases by a factor of [that predictor variable’s OR].

predict(wantsMorefit, type="response") # predicted values
##   1   2   3   4   5   6   7   8   9  10  11  12  13  14  15  16 
## 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5
residuals(wantsMorefit, type="deviance") # residuals
##        1        2        3        4        5        6        7        8 
##  1.17741 -1.17741  1.17741 -1.17741  1.17741 -1.17741  1.17741 -1.17741 
##        9       10       11       12       13       14       15       16 
##  1.17741 -1.17741  1.17741 -1.17741  1.17741 -1.17741  1.17741 -1.17741
plot(residuals(wantsMorefit, type="deviance")) # plot your residuals

detach(cuse)

Returning to our assumptions, we can now look at the residucals and plot them so we can see if they look weird. (They should look like a bunch of randomly scattered dots, not any special pattern.)

Above, our outcome variable was already entered as either 0 or 1. But what if your outcome isn’t already in standard 0/1 form? Here’s what you can do: Try working across columns! If you cbind the two columns into 1 vector, R can read one column as “hits” and the next as “misses” just like the previous analyses read the 0’s and 1’s. Here’s how we set up the columns like that. FYI, we copy a new column because I prefer not to overwrite the original data so I can’t use it later. Also, R reads the column in order as “hits” and then “misses,” so if your data isn’t already in that order, you may need to move or add columns to get them in that order. Here’s how.

Copy a Column in R 1. Create the new column

attach(cuse)
cuse$wantsMore_copy <- NA #This gives you a column of NAs. 
  1. Copy the data from the existing column into the new one.
cuse$wantsMore_copy <- cuse$wantsMore

cuse$wantsMore_copy<-recode(cuse$wantsMore_copy,"'no'=0;else=1") #Now those NAs have actual values.
  1. Run the model cbinding across the columns where the first column becomes “hits” and the second column becomes “misses.” You’ll see in our first line of the model, where the outcome variable usually appears, we have a cbind across our two columns instead. We have cbinded them into one variable for the purpose of this analysis.
usingfit <- glm(cbind(using, notUsing)~age+education+wantsMore_copy,data=cuse,family=binomial()) 
summary(usingfit) # display results
## 
## Call:
## glm(formula = cbind(using, notUsing) ~ age + education + wantsMore_copy, 
##     family = binomial(), data = cuse)
## 
## Deviance Residuals: 
##     Min       1Q   Median       3Q      Max  
## -2.5148  -0.9376   0.2408   0.9822   1.7333  
## 
## Coefficients:
##                 Estimate Std. Error z value Pr(>|z|)    
## (Intercept)      -0.8082     0.1590  -5.083 3.71e-07 ***
## age25-29          0.3894     0.1759   2.214  0.02681 *  
## age30-39          0.9086     0.1646   5.519 3.40e-08 ***
## age40-49          1.1892     0.2144   5.546 2.92e-08 ***
## educationlow     -0.3250     0.1240  -2.620  0.00879 ** 
## wantsMore_copy1  -0.8330     0.1175  -7.091 1.33e-12 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 165.772  on 15  degrees of freedom
## Residual deviance:  29.917  on 10  degrees of freedom
## AIC: 113.43
## 
## Number of Fisher Scoring iterations: 4
confint(usingfit) # 95% CI for the coefficients
## Waiting for profiling to be done...
##                       2.5 %      97.5 %
## (Intercept)     -1.12473993 -0.50071613
## age25-29         0.04632634  0.73646245
## age30-39         0.58955178  1.23552226
## age40-49         0.77093870  1.61220754
## educationlow    -0.56972464 -0.08329659
## wantsMore_copy1 -1.06373543 -0.60308458
exp(coef(usingfit)) # exponentiated coefficients
##     (Intercept)        age25-29        age30-39        age40-49 
##       0.4456506       1.4760678       2.4808804       3.2845805 
##    educationlow wantsMore_copy1 
##       0.7225312       0.4347628
exp(confint(usingfit)) # 95% CI for exponentiated coefficients
## Waiting for profiling to be done...
##                     2.5 %    97.5 %
## (Intercept)     0.3247369 0.6060965
## age25-29        1.0474162 2.0885341
## age30-39        1.8031800 3.4401747
## age40-49        2.1617946 5.0138673
## educationlow    0.5656812 0.9200782
## wantsMore_copy1 0.3451641 0.5471214
predict(usingfit, type="response") # predicted values
##         1         2         3         4         5         6         7 
## 0.1228009 0.2435683 0.1623053 0.3082699 0.1712509 0.3221665 0.2223899 
##         8         9        10        11        12        13        14 
## 0.3967948 0.2577768 0.4440846 0.3246329 0.5250773 0.3149818 0.5140024 
##        15        16 
## 0.3889006 0.5941188
residuals(usingfit, type="deviance") # residuals
##           1           2           3           4           5           6 
## -0.50713553  0.36079777  1.48747671 -2.51479462  0.40419450  0.25960953 
##           7           8           9          10          11          12 
##  1.22864003 -2.06525969 -0.84242507  1.64528938 -1.22297791  0.22193897 
##          13          14          15          16 
## -2.49141433 -0.06525423  0.90006915  1.73330325
plot(residuals(usingfit, type="deviance")) # plot your residuals

detach(cuse)

2 Logistic Regression Assumptions

Here are your assumptions: 1. Independence. Just know your data, know your sampling methods. Do they make a decent case for independence? Yes? Godspeed, classmates. Onto assumption #2. 2. Non-fishy distribution of the residuals. (See residuals plot above.) 3. Correct specification of the variance structure 4. Linear relationship between the response and the linear predictor
For 2-4, you need to use your model. Also, your predictor may not be linear, so don’t be a perfectionist. Let’s get into the analysis, then.

In mydata, our outcome variable is admit (i.e., admission to grad school, which is a yes/no response.) We’ve also got predictor variables: gre (i.e., GRE exam score) and gpa (undergraduate GPA), which are continuous(ish), and rank (ranking of undergrad institution where 1 is the best and 4 is the worst, sort of like tiers) which is more ordinal/categorical as a variable. This dataset is asking whether and to what extent GPA, GRE score, and rank of institution changes the likelihood of getting admitted to grad school.

Let’s take a look at our data and make sure we in fact have a binary outcome. Certain graphs and analyses require continuous vs. categorical predictors, so we can confirm what we’re working with by getting a summary. Knowing our data should help us cover the 1st assumption. This will use the data from and modify/add onto the UCLA Statistical Consulting Group’s tutorial. Their dataset is different from cuse and allows us to do different logistic analyses.

## The following object is masked from package:pscl:
## 
##     admit
##      admit             gre             gpa             rank      
##  Min.   :0.0000   Min.   :220.0   Min.   :2.260   Min.   :1.000  
##  1st Qu.:0.0000   1st Qu.:520.0   1st Qu.:3.130   1st Qu.:2.000  
##  Median :0.0000   Median :580.0   Median :3.395   Median :2.000  
##  Mean   :0.3175   Mean   :587.7   Mean   :3.390   Mean   :2.485  
##  3rd Qu.:1.0000   3rd Qu.:660.0   3rd Qu.:3.670   3rd Qu.:3.000  
##  Max.   :1.0000   Max.   :800.0   Max.   :4.000   Max.   :4.000
##       admit         gre         gpa        rank 
##   0.4660867 115.5165364   0.3805668   0.9444602

Here are some ways to eyeball the data for your assumptions. Is your outcome variable dichotomous? There should only be 2 possible outcomes (or levels) for that variable. Visualizing your data in a table will show you how many levels you’re dealing with. Take a look: Since admit (admission to grad school, yes or no) is our outcome variable here, does it show you exactly 2 levels of admit in the table? If yes, that’s a point toward meeting our assumptions.

xtabs(~ admit + rank, data = mydata) 
##      rank
## admit  1  2  3  4
##     0 28 97 93 55
##     1 33 54 28 12

A conditional density plot will also show us if our distribution looks binary. You’re looking for a diagonal(ish) plot with odds that differ according to the level (0 or 1) of your outcome variable.

What if it isn’t reading the DV as a factor? Earlier it gave us a mean on this, so it’s probably reading it as a numeric value, not as a factor, like it needs to be. Let’s make a 2nd version of admit that R will definitely read as a factor with 2 levels. Keep this in mind: it’s a very common error in logistic regression in R. Here’s how to remind R that it’s definitely a factor:

mydata$admit2 = factor(mydata$admit)

We tell R to add a variable called admit2 to mydata, which is a factored version of our original admit.

levels(mydata$admit2) = c("no","yes") 

And just to make sure it’s not read as a numeric again, let’s put some non-numeric levels on admit2. If you click on mydata in the Global Environment, you should see admit2 as a column with yes and no corresponding to the 0’s and 1’s in admit.

Reminder: 0 = no, 1 = yes.

Now let’s try that conditional density plot with admit2.

2.1 Logistic Regression and Categorical Predictor Variables that are Also Ordinal

Above we ran a basic glm model and looked at main effects. Here’s how we can do some other analyses with our grad school admit data. Here’s a cool categorical analysis from the UCLA Logit Regression Tutorial that I think is worth knowing you can do with a logistic model. Here, rank has an order and we can analyze the potential change in the outcome (admission to grad school) depending on each level of rank.

## 
## Call:
## glm(formula = admit ~ gre + gpa + rank, family = "binomial", 
##     data = mydata)
## 
## Deviance Residuals: 
##     Min       1Q   Median       3Q      Max  
## -1.6268  -0.8662  -0.6388   1.1490   2.0790  
## 
## Coefficients:
##              Estimate Std. Error z value Pr(>|z|)    
## (Intercept) -3.989979   1.139951  -3.500 0.000465 ***
## gre          0.002264   0.001094   2.070 0.038465 *  
## gpa          0.804038   0.331819   2.423 0.015388 *  
## rank2       -0.675443   0.316490  -2.134 0.032829 *  
## rank3       -1.340204   0.345306  -3.881 0.000104 ***
## rank4       -1.551464   0.417832  -3.713 0.000205 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 499.98  on 399  degrees of freedom
## Residual deviance: 458.52  on 394  degrees of freedom
## AIC: 470.52
## 
## Number of Fisher Scoring iterations: 4
## Waiting for profiling to be done...
##                     2.5 %       97.5 %
## (Intercept) -6.2716202334 -1.792547080
## gre          0.0001375921  0.004435874
## gpa          0.1602959439  1.464142727
## rank2       -1.3008888002 -0.056745722
## rank3       -2.0276713127 -0.670372346
## rank4       -2.4000265384 -0.753542605
##                     2.5 %       97.5 %
## (Intercept) -6.2242418514 -1.755716295
## gre          0.0001202298  0.004408622
## gpa          0.1536836760  1.454391423
## rank2       -1.2957512650 -0.055134591
## rank3       -2.0169920597 -0.663415773
## rank4       -2.3703986294 -0.732528724
## Wald test:
## ----------
## 
## Chi-squared test:
## X2 = 20.9, df = 3, P(> X2) = 0.00011

The p-value in the above wald test tells you if the overall effect of that categorical variable is significant.

This is almost akin to running an omnibus test in ANOVA. We know something is happening with rank, so here’s how you can compare levels of rank.

## Wald test:
## ----------
## 
## Chi-squared test:
## X2 = 5.5, df = 1, P(> X2) = 0.019
## (Intercept)         gre         gpa       rank2       rank3       rank4 
##   0.0185001   1.0022670   2.2345448   0.5089310   0.2617923   0.2119375
## Waiting for profiling to be done...
##                    OR       2.5 %    97.5 %
## (Intercept) 0.0185001 0.001889165 0.1665354
## gre         1.0022670 1.000137602 1.0044457
## gpa         2.2345448 1.173858216 4.3238349
## rank2       0.5089310 0.272289674 0.9448343
## rank3       0.2617923 0.131641717 0.5115181
## rank4       0.2119375 0.090715546 0.4706961

2.2 Here Are the Rest of your Stats to Look at Interactions

attach(cuse)
Intusingfit <- glm(cbind(using, notUsing)~age*education,data=cuse,family=binomial()) 
summary(Intusingfit) # display results
## 
## Call:
## glm(formula = cbind(using, notUsing) ~ age * education, family = binomial(), 
##     data = cuse)
## 
## Deviance Residuals: 
##     Min       1Q   Median       3Q      Max  
## -3.7849  -1.0158  -0.0602   1.3826   3.4456  
## 
## Coefficients:
##                       Estimate Std. Error z value Pr(>|z|)    
## (Intercept)            -1.4412     0.1412 -10.205  < 2e-16 ***
## age25-29                0.4420     0.1919   2.303   0.0213 *  
## age30-39                1.0357     0.1827   5.668 1.44e-08 ***
## age40-49                2.1090     0.3092   6.822 9.00e-12 ***
## educationlow           -0.3993     0.3685  -1.084   0.2785    
## age25-29:educationlow   0.2071     0.4550   0.455   0.6490    
## age30-39:educationlow   0.2904     0.4042   0.719   0.4724    
## age40-49:educationlow  -0.6740     0.4923  -1.369   0.1710    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 165.772  on 15  degrees of freedom
## Residual deviance:  73.033  on  8  degrees of freedom
## AIC: 160.54
## 
## Number of Fisher Scoring iterations: 4
confint(Intusingfit) # 95% CI for the coefficients
## Waiting for profiling to be done...
##                             2.5 %     97.5 %
## (Intercept)           -1.72642022 -1.1718125
## age25-29               0.06731008  0.8207309
## age30-39               0.68123554  1.3983248
## age40-49               1.51470767  2.7315906
## educationlow          -1.17430407  0.2846155
## age25-29:educationlow -0.66503892  1.1304095
## age30-39:educationlow -0.47008655  1.1267953
## age40-49:educationlow -1.62388932  0.3165131
exp(coef(Intusingfit)) # exponentiated coefficients
##           (Intercept)              age25-29              age30-39 
##             0.2366412             1.5558651             2.8172043 
##              age40-49          educationlow age25-29:educationlow 
##             8.2403226             0.6707629             1.2301350 
## age30-39:educationlow age40-49:educationlow 
##             1.3370229             0.5096888
exp(confint(Intusingfit)) # 95% CI for exponentiated coefficients
## Waiting for profiling to be done...
##                           2.5 %     97.5 %
## (Intercept)           0.1779202  0.3098049
## age25-29              1.0696271  2.2721599
## age30-39              1.9763180  4.0484125
## age40-49              4.5480914 15.3572950
## educationlow          0.3090340  1.3292509
## age25-29:educationlow 0.5142535  3.0969244
## age30-39:educationlow 0.6249482  3.0857517
## age40-49:educationlow 0.1971305  1.3723342
predict(Intusingfit, type="response") # predicted values
##         1         2         3         4         5         6         7 
## 0.1369863 0.1369863 0.1913580 0.1913580 0.2330097 0.2330097 0.2691030 
##         8         9        10        11        12        13        14 
## 0.2691030 0.3741722 0.3741722 0.4000000 0.4000000 0.4000000 0.4000000 
##        15        16 
## 0.6610169 0.6610169
residuals(Intusingfit, type="deviance") # residuals
##          1          2          3          4          5          6 
## -0.8207009  1.4484269  0.2309360 -0.4947108 -0.9132139  1.3606657 
##          7          8          9         10         11         12 
## -0.3512671  0.5223897 -3.7848869  3.4455922 -3.1959327  3.2711054 
##         13         14         15         16 
## -3.5518799  2.1658149 -1.3235504  0.8435519
1-pchisq(73.033,8) # 1 - chisquare (residual deviance, DF)
## [1] 1.220468e-12
#Analysis of Deviance Table
anova(Intusingfit, test="Chisq") 
## Analysis of Deviance Table
## 
## Model: binomial, link: logit
## 
## Response: cbind(using, notUsing)
## 
## Terms added sequentially (first to last)
## 
## 
##               Df Deviance Resid. Df Resid. Dev Pr(>Chi)    
## NULL                             15    165.772             
## age            3   79.192        12     86.581  < 2e-16 ***
## education      1    6.162        11     80.418  0.01305 *  
## age:education  3    7.385         8     73.033  0.06057 .  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
#McFadden R^2 is a pseudo R^2 for logistic regression
pR2(Intusingfit)
##          llh      llhNull           G2     McFadden         r2ML 
##  -72.2703401 -118.6401419   92.7396036    0.3908441    0.9969610 
##         r2CU 
##    0.9969614

2.3 Updating the Model

What if you (or your PI) suddenly decide you want to see the analysis, but WITHOUT that one predictor variable? Fret not, you don’t need to rewrite the whole thing. You can use the update function instead. Since it’s easier to demonstrate this with a simple additive model, we’ll look at it with the old usingfit one, which only looked at main effects and not interactions. This is the model from the 3rd section of “Copying a Column in R.” Now we’ll look at it without education.

Updateusingfit <- update(usingfit, ~ . - education) 
summary(Updateusingfit)
## 
## Call:
## glm(formula = cbind(using, notUsing) ~ age + wantsMore_copy, 
##     family = binomial(), data = cuse)
## 
## Deviance Residuals: 
##     Min       1Q   Median       3Q      Max  
## -2.7870  -1.3208  -0.3417   1.2346   2.4577  
## 
## Coefficients:
##                 Estimate Std. Error z value Pr(>|z|)    
## (Intercept)      -0.8698     0.1571  -5.536 3.10e-08 ***
## age25-29          0.3678     0.1754   2.097    0.036 *  
## age30-39          0.8078     0.1598   5.056 4.27e-07 ***
## age40-49          1.0226     0.2039   5.014 5.32e-07 ***
## wantsMore_copy1  -0.8241     0.1171  -7.037 1.97e-12 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 165.772  on 15  degrees of freedom
## Residual deviance:  36.888  on 11  degrees of freedom
## AIC: 118.4
## 
## Number of Fisher Scoring iterations: 4
detach(cuse)

Regarding the McFadden R^2, which is a pseudo R^2 for logistic regression…A regular (i.e., non-pseudo) R^2 in ordinary least squares regression is often used as an indicator of goodness-of-fit. However, in a logistic regression we don’t have the types of values to calculate a real R^2. A pseudo R^2 here is calculated differently and is similarly used as a measure of goodness of fit (lower is better.) But that’s not all there is to pseudo-R^2’s. Here’s a good explanation with more detail.

2.4 Graphing Logistic Regression Analyses

This takes some work. Because we talk about binary outcomes in terms of odds ratios, but it’s not very informative to graph odds ratios, we need to work instead with predicted probabilities and graph those. That involves making a bunch of new datasets. This is my annotated version of the UCLA tutorial for beginners, along with titling the graph because untitled graphs are my pet peeve.

  1. Start by calculating the predicted probability of the outcome at each level of the categorical/ordinal predictor variable and/or holding continuous variables at their means.
attach(mydata)
## The following object is masked from package:pscl:
## 
##     admit
newdata1 <- with(mydata,
                 data.frame(gre = mean(gre), gpa = mean(gpa), rank = factor(1:4)))
  1. Create the predicted probabilities using the new dataframe
newdata1$rankP <- predict(mylogit, newdata = newdata1, type = "response")
newdata1
##     gre    gpa rank     rankP
## 1 587.7 3.3899    1 0.5166016
## 2 587.7 3.3899    2 0.3522846
## 3 587.7 3.3899    3 0.2186120
## 4 587.7 3.3899    4 0.1846684

Note: This actually requires using rep and seq functions in order to make our data fit our GRE score range and have tic marks at 100 points, similar to what our SD showed us about the GRE scores’ distribution. We also have to constantly remind R that our factors are indeed factors. If you have something logistic that won’t run, this is a common reason why.

  1. Start creating the predicted probability version of your variables
newdata2 <- with(mydata,
                 data.frame(gre = rep(seq(from = 200, to = 800, length.out = 100), 4),
                            gpa = mean(gpa), rank = factor(rep(1:4, each = 100))))
  1. Set the SE to display
newdata3 <- cbind(newdata2, predict(mylogit, newdata = newdata2, type="link", se=TRUE))
newdata3 <- within(newdata3, {
  PredictedProb <- plogis(fit)
  LL <- plogis(fit - (1.96 * se.fit))
  UL <- plogis(fit + (1.96 * se.fit))
})
  1. Finally, the graph
PredProbPlot <- ggplot(newdata3, aes(x = gre, y = PredictedProb)) +
  geom_ribbon(aes(ymin = LL, ymax = UL, fill = rank), alpha = .2) +
  geom_line(aes(colour = rank), size=1)

PredProbPlot + ggtitle("Admission to Grad School by Rank") #add a title

detach(mydata)

Interpreting the graph: Each line is the predicted probability of being admitted to grad school for each institutional rank. The legend tells us that red represents #1 ranked institutions, green represents #2 ranked institutions, blue represents #3 ranked institutions, and purple represents #4 ranked institutions (in order from top ranked to lowest ranked.) Each is set agaist the color-coded confidence interval. We can see that the intercepts all fall in rank order and that for each institution rank, the positive slope shows the predicted probability of being admitted to grad school increases as GRE score increases.

Plotting an Interaction Using the Effects Package

attach(cuse)
AgeWantfit <- glm(cbind(using,notUsing)~
                    age*wantsMore_copy,data=cuse,
                  family=binomial())
summary(AgeWantfit) # display results
## 
## Call:
## glm(formula = cbind(using, notUsing) ~ age * wantsMore_copy, 
##     family = binomial(), data = cuse)
## 
## Deviance Residuals: 
##      Min        1Q    Median        3Q       Max  
## -1.67001  -0.85288   0.02621   0.72300   2.18925  
## 
## Coefficients:
##                          Estimate Std. Error z value Pr(>|z|)    
## (Intercept)               -1.4553     0.2968  -4.903 9.43e-07 ***
## age25-29                   0.6354     0.3564   1.783  0.07463 .  
## age30-39                   1.5412     0.3183   4.842 1.29e-06 ***
## age40-49                   1.7643     0.3435   5.136 2.80e-07 ***
## wantsMore_copy1           -0.0640     0.3303  -0.194  0.84637    
## age25-29:wantsMore_copy1  -0.2672     0.4091  -0.653  0.51366    
## age30-39:wantsMore_copy1  -1.0905     0.3733  -2.921  0.00349 ** 
## age40-49:wantsMore_copy1  -1.3672     0.4834  -2.828  0.00468 ** 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 165.772  on 15  degrees of freedom
## Residual deviance:  20.099  on  8  degrees of freedom
## AIC: 107.61
## 
## Number of Fisher Scoring iterations: 4
plot(effect("age:wantsMore_copy",
            AgeWantfit,
            xlevels = list(age=4, wantsMore_copy=c(1,0))),
     rug=F,
     multiline=T,
     main="Age by Wants More Interaction \n to Predict Using Contraceptives",
     ylab="Contraceptive Use",
     xlab="Age")

detach(cuse)

Interpreting the graph: The legend tells us that the black line with circles represents women who DON’T want more kids and the red dashed line with triangles represents women who DO want more kids. We see that, in general across both groups, the older women are, the more likely they are to use contraceptives. However, we see an Group by Age interaction. The slope is much steeper for women who DON’T want more kids: they are markedly more likely to use contraceptives than women who DO want more kids above. Women who DO want more kids experience less of an impact of age on their contraceptive use than do women who DON’T want more kids.

3 Poisson Regression

3.1 That, but Change the family to “Poisson”

No, but seriously, here’s the entire Poisson section on Robert I. Kabacoff’s quickR blog at http://www.statmethods.net/advstats/glm.html:

#Poisson Regression
#where count is a count and 
# x1-x3 are continuous predictors 
#fit <- glm(count ~ x1+x2+x3, data=mydata, family=poisson())
#summary(fit) display results

See? I wasn’t joking. But let’s do one for real. Here’s data I got from http://www.theanalysisfactor.com/generalized-linear-models-in-r-part-6-poisson-regression-count-variables/ by David Lillis.

It’s about cases (i.e., counts) of disease among high school students by number of days after outbreak. Here’s the data, called ‘cases.’ Each time, run the whole chunk at once or it won’t work.

cases <-  
structure(list(Days = c(1L, 2L, 3L, 3L, 4L, 4L, 4L, 6L, 7L, 8L, 
8L, 8L, 8L, 12L, 14L, 15L, 17L, 17L, 17L, 18L, 19L, 19L, 20L, 
23L, 23L, 23L, 24L, 24L, 25L, 26L, 27L, 28L, 29L, 34L, 36L, 36L, 
42L, 42L, 43L, 43L, 44L, 44L, 44L, 44L, 45L, 46L, 48L, 48L, 49L, 
49L, 53L, 53L, 53L, 54L, 55L, 56L, 56L, 58L, 60L, 63L, 65L, 67L, 
67L, 68L, 71L, 71L, 72L, 72L, 72L, 73L, 74L, 74L, 74L, 75L, 75L, 
80L, 81L, 81L, 81L, 81L, 88L, 88L, 90L, 93L, 93L, 94L, 95L, 95L, 
95L, 96L, 96L, 97L, 98L, 100L, 101L, 102L, 103L, 104L, 105L, 
106L, 107L, 108L, 109L, 110L, 111L, 112L, 113L, 114L, 115L), 
    Students = c(6L, 8L, 12L, 9L, 3L, 3L, 11L, 5L, 7L, 3L, 8L, 
    4L, 6L, 8L, 3L, 6L, 3L, 2L, 2L, 6L, 3L, 7L, 7L, 2L, 2L, 8L, 
    3L, 6L, 5L, 7L, 6L, 4L, 4L, 3L, 3L, 5L, 3L, 3L, 3L, 5L, 3L, 
    5L, 6L, 3L, 3L, 3L, 3L, 2L, 3L, 1L, 3L, 3L, 5L, 4L, 4L, 3L, 
    5L, 4L, 3L, 5L, 3L, 4L, 2L, 3L, 3L, 1L, 3L, 2L, 5L, 4L, 3L, 
    0L, 3L, 3L, 4L, 0L, 3L, 3L, 4L, 0L, 2L, 2L, 1L, 1L, 2L, 0L, 
    2L, 1L, 1L, 0L, 0L, 1L, 1L, 2L, 2L, 1L, 1L, 1L, 1L, 0L, 0L, 
    0L, 1L, 1L, 0L, 0L, 0L, 0L, 0L)), .Names = c("Days", "Students"
), class = "data.frame", row.names = c(NA, -109L))

attach(cases)

Behold, the Poisson distribution! Notice the mode at one extreme and the tail off to the other side. This is characteristic of Poisson distributions.

plot(Days, Students, xlab = "Days", ylab = "Students", pch = 16)
title(main = "Spread of Teenage Disease Post-Outbreak", sub = NULL,
      line = NA, outer = FALSE) 

3.2 Seriously, Just Change Family

PoissonModel <- glm(Students ~ Days, poisson)
summary(PoissonModel)
## 
## Call:
## glm(formula = Students ~ Days, family = poisson)
## 
## Deviance Residuals: 
##      Min        1Q    Median        3Q       Max  
## -2.00482  -0.85719  -0.09331   0.63969   1.73696  
## 
## Coefficients:
##              Estimate Std. Error z value Pr(>|z|)    
## (Intercept)  1.990235   0.083935   23.71   <2e-16 ***
## Days        -0.017463   0.001727  -10.11   <2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for poisson family taken to be 1)
## 
##     Null deviance: 215.36  on 108  degrees of freedom
## Residual deviance: 101.17  on 107  degrees of freedom
## AIC: 393.11
## 
## Number of Fisher Scoring iterations: 5
exp(coef(PoissonModel))
## (Intercept)        Days 
##   7.3172529   0.9826884
exp(confint(PoissonModel))
## Waiting for profiling to be done...
##                2.5 %    97.5 %
## (Intercept) 6.188427 8.6004621
## Days        0.979322 0.9859775

Rather than odds ratios (which only apply to 0/1 outcomes), we use relative risk ratios in Poisson regression for count outcome variables.

Interpretation:The expected log count for each unit increase/decrease (depending on the sign of the coefficient) in [outcome variable] given [predictor variable] is [coefficient].

4 References

Kabacoff, R.I. Generalized Linear Models. Retrieved from http://www.statmethods.net/advstats/glm.html Lillis, D. Generalized Linear Models in R, Part 6: Poisson Regression for Count Variables. Retrieved from http://www.theanalysisfactor.com/generalized-linear-models-in-r-part-6-poisson-regression-count-variables/
Rodríguez, G. 5 Generalized Linear Models. Retrieved from http://data.princeton.edu/R/glms.html UCLA: Statistical Consulting Group. R Data Analysis Examples: Logit Regression. Retrieved from http://www.ats.ucla.edu/stat/r/dae/logit.htm

LS0tDQp0aXRsZTogJ0NoYXB0ZXIgMTk6IExvZ2lzdGljIGFuZCBQb2lzc29uIFJlZ3Jlc3Npb24nDQphdXRob3I6ICJNYXJpZSBDaGVzYW5pdWsiDQpvdXRwdXQ6DQogIGh0bWxfZG9jdW1lbnQ6DQogICAgdGhlbWU6IGNlcnVsZWFuDQogICAgaGlnaGxpZ2h0OiB0ZXh0bWF0ZQ0KICAgIGZvbnRzaXplOiA4cHQNCiAgICB0b2M6IHRydWUNCiAgICBudW1iZXJfc2VjdGlvbnM6IHRydWUNCiAgICBjb2RlX2Rvd25sb2FkOiB0cnVlDQogICAgdG9jX2Zsb2F0Og0KICAgICAgY29sbGFwc2VkOiBmYWxzZQ0KLS0tDQoNCiMgTG9naXN0aWMgJiBQb2lzc29uIFJlZ3Jlc3Npb246IE92ZXJ2aWV3DQpJbiB0aGlzIGNoYXB0ZXIsIEkndmUgbWFzaGVkIHRvZ2V0aGVyIG9ubGluZSBkYXRhc2V0cywgdHV0b3JpYWxzLCBhbmQgbXkgb3duIG1vZGlmaWNhdGlvbnMgdGhlcmV0by4gSSBzdGFydCB3aXRoIHRoZSBwYWNrYWdlcyB3ZSB3aWxsIG5lZWQuIFRoZW4gSSBtb3ZlIGludG8gZGF0YSBjbGVhbmluZyBhbmQgYXNzdW1wdGlvbnMuIFRoZSBtb2RlbCBpdHNlbGYgaXMgcG9zc2libHkgdGhlIGVhc2llc3QgdGhpbmcgdG8gcnVuLiBUaGVuLCB3ZSB3cmFwIHVwIHdpdGggYWxsIHRoZSBzdGF0cyB5b3UnbGwgZXZlciBuZWVkIGZvciB5b3VyIGxvZ2lzdGljIHJlZ3Jlc3Npb24gYW5kIGhvdyB0byBncmFwaCBpdC4gQmVmb3JlIHdlIGxlYXZlLCB3ZSdsbCBsb29rIGF0IHRoZSBzbGlnaHQgbW9kaWZpY2F0aW9uIGZvciBydW5uaW5nIGEgUG9pc3NvbiByZWdyZXNzaW9uLiAgDQoNCiMjIFdoeSB3b3VsZCB5b3UgZG8gbG9naXN0aWMgcmVncmVzc2lvbj8NCkxvZ2lzdGljIHJlZ3Jlc3Npb24sIGFsc28ga25vd24gYXMgbG9naXQgcmVncmVzc2lvbiwgaXMgd2hhdCB5b3UgdXNlIHdoZW4geW91ciBvdXRjb21lIHZhcmlhYmxlIChkZXBlbmRlbnQgdmFyaWFibGUpIGlzIGRpY2hvdG9tb3VzLiBUaGVzZSB3b3VsZCByZWZlciB0byBhbGwgeW91ciByZXNlYXJjaCB5ZXMvbm8gcXVlc3Rpb25zOiBEaWQgeW91IHZvdGUsIHllcyBvciBubz8gRG8geW91IGhhdmUgYSBkaXNlYXNlLCB5ZXMgb3Igbm8/IERpZCB5b3UgdXNlIFtpbnNlcnQgeW91ciBmYXZvcml0ZSBpbGxpY2l0IHN1YnN0YW5jZSBoZXJlXSBpbiB0aGUgcGFzdCB3ZWVrLCB5ZXMgb3Igbm8/IFlvdSBtaWdodCBhbHNvIHRoaW5rIG9mIGl0IGFzIHBhc3MvZmFpbCwgaGl0L21pc3MsIHN1Y2Nlc3MvZmFpbHVyZSwgYWxpdmUvZGVhZCwgZXRjLiBSYXRoZXIgdGhhbiBlc3RpbWF0ZSBiZXRhIHNpemVzLCB0aGUgbG9naXN0aWMgcmVncmVzc2lvbiBlc3RpbWF0ZXMgdGhlIHByb2JhYmlsaXR5IG9mIGdldHRpbmcgb25lIG9mIHlvdXIgdHdvIG91dGNvbWVzIChpLmUuLCB0aGUgcHJvYmFiaWxpdHkgb2Ygdm90aW5nIHZzLiBub3Qgdm90aW5nKSBnaXZlbiBhIHByZWRpY3Rvci9pbmRlcGVuZGVudCB2YXJpYWJsZShzKS4gRm9yIG91ciBwdXJwb3NlcywgImhpdCIgcmVmZXJzIHRvIHlvdXIgZmF2b3JlZCBvdXRjb21lIGFuZCAibWlzcyIgcmVmZXJzIHRvIHlvdXIgdW5mYXZvcmVkIG91dGNvbWUuDQoNCiMjIFdoeSB3b3VsZCB5b3UgZG8gYSBQb2lzc29uIHJlZ3Jlc3Npb24/DQpQb2lzc29uIHJlZ3Jlc3Npb24sIGFsc28ga25vd24gYXMgYSBsb2ctbGluZWFyIG1vZGVsLCBpcyB3aGF0IHlvdSB1c2Ugd2hlbiB5b3VyIG91dGNvbWUgdmFyaWFibGUgaXMgYSBjb3VudCAoaS5lLiwgbnVtZXJpYywgYnV0IG5vdCBxdWl0ZSBzbyB3aWRlIGluIHJhbmdlIGFzIGEgY29udGludW91cyB2YXJpYWJsZS4pIEV4YW1wbGVzIG9mIGNvdW50IHZhcmlhYmxlcyBpbiByZXNlYXJjaCBpbmNsdWRlIGhvdyBtYW55IGhlYXJ0IGF0dGFja3Mgb3Igc3Ryb2tlcyBvbmUncyBoYWQsIGhvdyBtYW55IGRheXMgaW4gdGhlIHBhc3QgbW9udGggb25lJ3MgdXNlZCBbaW5zZXJ0IHlvdXIgZmF2b3JpdGUgaWxsaWNpdCBzdWJzdGFuY2UgaGVyZV0sIG9yLCBhcyBpbiBzdXJ2aXZhbCBhbmFseXNpcywgaG93IG1hbnkgZGF5cyBmcm9tIG91dGJyZWFrIHVudGlsIGluZmVjdGlvbi4gVGhlIFBvaXNzb24gZGlzdHJpYnV0aW9uIGlzIHVuaXF1ZSBpbiB0aGF0IGl0cyBtZWFuIGFuZCBpdHMgdmFyaWFuY2UgYXJlIGVxdWFsLiBUaGlzIGlzIG9mdGVuIGR1ZSB0byB6ZXJvIGluZmxhdGlvbi4gU29tZXRpbWVzIHR3byBwcm9jZXNzZXMgbWF5IGJlIGF0IHdvcms6IG9uZSB0aGF0IGRldGVybWluZXMgd2hldGhlciBvciBub3QgYW4gZXZlbnQgaGFwcGVucyBhdCBhbGwgYW5kIGFub3RoZXIgdGhhdCBkZXRlcm1pbmVzIGhvdyBtYW55IHRpbWVzIHRoZSBldmVudCBoYXBwZW5zIHdoZW4gaXQgZG9lcy4gVXNpbmcgb3VyIGNvdW50IHZhcmlhYmxlcyBmcm9tIGFib3ZlLCB0aGlzIGNvdWxkIGJlIGEgc2FtcGxlIHRoYXQgY29udGFpbnMgaW5kaXZpZHVhbHMgd2l0aCBhbmQgd2l0aG91dCBoZWFydCBkaXNlYXNlOiB0aG9zZSB3aXRob3V0IGhlYXJ0IGRpc2Vhc2UgY2F1c2UgYSBkaXNwcm9wb3J0aW9uYXRlIGFtb3VudCBvZiB6ZXJvcyBpbiB0aGUgZGF0YSBhbmQgdGhvc2Ugd2l0aCBoZWFydCBkaXNlYXNlIHRyYWlsIG9mZiBpbiBhIHRhaWwgdG8gdGhlIHJpZ2h0IHdpdGggaW5jcmVhc2luZyBhbW91bnRzIG9mIGhlYXJ0IGF0dGFja3MuIFRoaXMgaXMgd2h5IGxvZ2lzdGljIGFuZCBQb2lzc29uIHJlZ3Jlc3Npb25zIGdvIHRvZ2V0aGVyIGluIHJlc2VhcmNoOiB0aGVyZSBpcyBhIGRpY2hvdG9tb3VzIG91dGNvbWUgaW5oZXJlbnQgaW4gYSBQb2lzc29uIGRpc3RyaWJ1dGlvbi4gSG93ZXZlciwgdGhlICJoaXRzIiBpbiB0aGUgbG9naXN0aWMgcXVlc3Rpb24gY2FuJ3QgYmUgdW5kZXJzdG9vZCB3aXRob3V0IGZ1cnRoZXIgY29uZHVjdGluZyB0aGUgUG9pc3NvbiByZWdyZXNzaW9uLg0KDQojIyMgUGFja2FnZXMNCldlJ3JlIGdvaW5nIHRvIHN0YXJ0IGJ5IGxvYWRpbmcgdGhlIGZvbGxvd2luZyBsaWJyYXJpZXMuIEhlcmUncyB3aHk6DQoNCiogX19hb2Q6X18gU3RhbmRzIGZvciBBbmFseXNpcyBvZiBPdmVyZGlzcGVyc2VkIERhdGEsIGNvbnRhaW5zIGEgYnVuY2ggb2YgZnVuY3Rpb25zIGZvciB0aGlzIHR5cGUgb2YgYW5hbHlzaXMgb24gY291bnRzIG9yIHByb3BvcnRpb25zLiBUaGUgb25lIHlvdSdsbCBzZWUgdGhlIG1vc3QgaW4gdGhpcyBjaGFwdGVyIGlzIHdhbGQudGVzdCAoaS5lLiwgV2FsZCBUZXN0IGZvciBNb2RlbCBDb2VmZmljaWVudHMuKQ0KDQoqIF9fZ2dwbG90MjpfXyBnZ3Bsb3QyIGlzIGFuIGV4cGFuc2lvbiBvbiBMZWxhbmQgV2lsa2luc29uJ3MgVGhlIEdyYW1tYXIgb2YgR3JhcGhpY3MuIGdncGxvdDIgbGV0J3MgeW91IGdyYXBoaWNhbGx5IHJlcHJlc2VudCBib3RoIHVuaXZhcmlhdGUgYW5kIG11bHRpdmFyaWF0ZSBudW1lcmljYWwgYW5kIGNhdGVnb3JpY2FsIGRhdGEuIFNpbmNlIHdlJ3JlIGRvaW5nIGxvZ2lzdGljIHJlZ3Jlc3Npb24sIHdlIG5lZWQgYSBncmFwaGluZyBsaWJyYXJ5IHRoYXQgY2FuIGhhbmRsZSBjYXRlZ29yaWNhbCBkYXRhLg0KDQoqIF9fZWZmZWN0czpfXyBBbm90aGVyIGdyYXBoaW5nIHBhY2thZ2UgYmVjYXVzZSBvcHRpb25zIGFyZSBnb29kLg0KDQoqIF9fZ3JhcGhpY3M6X18gVGhpcyBwYWNrYWdlIGFsbG93cyB5b3UgdG8gZ28gYmV5b25kIFIgZ3JhcGhpbmcgcHJpbWl0aXZlcy4gQmVsb3csIHlvdSdsbCBzZWUgY2RwbG90CShDb25kaXRpb25hbCBEZW5zaXR5IFBsb3RzKSwgeGxpbSwgeWxpbSwgYW5kIG90aGVycyBpbiBhY3Rpb24uIA0KDQoqIF9fY2FyOl9fIExldHMgdXMgbW9kaWZ5IHNpbXBsZSBwbG90cyBpbiBSIHdpdGggbGFiZWxzLCB0aXRsZXMsIGV0Yy4gIA0KDQoqIF9fcHNjbDpfXyBOZWVkIHRoaXMgdG8gY3JlYXRlIGEgcHNldWRvIFItc3F1YXJlZCBmb3IgbG9naXN0aWMgcmVncmVzc2lvbi4gIA0KDQpfTm90ZTpfIFlvdSBtaWdodCBnZXQgc29tZSBXYXJuaW5nIE1lc3NhZ2VzIHdoZW4geW91IGxvYWQgdGhlc2UgZGVwZW5kaW5nIG9uIHdoYXQgdmVyc2lvbiBvZiBSU3R1ZGlvIHlvdSBoYXZlIGFuZC9vciB3aGV0aGVyIHlvdSd2ZSBwcmV2aW91c2x5IGxvYWRlZCB0aGVzZSBpbiB0aGUgcGFzdC4gWW91IG1heSBhbHNvIG5lZWQgdG8gZ28gdG8gaW50byBQYWNrYWdlcz5JbnN0YWxsIGFuZCBjaGVjayB0aGUgYm94IG5leHQgdG8gdGhlIHBhY2thZ2UgeW91IHdhbnQgdG8gbG9hZCBpZiB5b3UncmUgdXNpbmcgUlN0dWRpby4gVXN1YWxseSwgb25jZSB5b3UgaW5zdGFsbCBhIHBhY2thZ2UsIGxpYnJhcnkoKSBzaG91bGQgYmUgc3VmZmljaWVudCB0byBsb2FkIGl0IGluIHRoZSBmdXR1cmUuDQpgYGB7ciBlY2hvPUZBTFNFfQ0KbGlicmFyeShhb2QpDQpsaWJyYXJ5KGdncGxvdDIpDQpsaWJyYXJ5KGVmZmVjdHMpDQpsaWJyYXJ5KGdyYXBoaWNzKQ0KbGlicmFyeShjYXIpDQpsaWJyYXJ5KHBzY2wpDQpgYGANCiAgICAgDQojIyBOb3cgbGV0J3MgbG9hZCBvdXIgZGF0YS4gICANCg0KSSdsbCBiZSBicmluZ2luZyBpbiBhIGNvdXBsZSBkYXRhc2V0cyBmcmVlbHkgYXZhaWxhYmxlIG9ubGluZSBpbiBvcmRlciB0byBkZW1vbnN0cmF0ZSB3aGF0IG5lZWRzIHRvIGhhcHBlbiBpbiBsb2dpc3RpYyByZWdyZXNzaW9uLiBJIGJhc2ljYWxseSBzdHJpbmcgdG9nZXRoZXIgdGhpbmdzIGF2YWlsYWJsZSBpbiBzZXZlcmFsIHBsYWNlcyBvbmxpbmUgc28gdGhhdCB3ZSBoYXZlIGV2ZXJ5dGhpbmcgd2UgbmVlZCBmb3IgbG9naXN0aWMgcmVncmVzc2lvbiBhbmFseXNpcyBoZXJlIGluIG9uZSBjaGFwdGVyLiBTZWUgZW5kbm90ZXMgZm9yIGxpbmtzIGFuZCByZWZlcmVuY2VzLiBXZSBoYXZlIDIgZGF0YXNldHMgd2UnbGwgYmUgd29ya2luZyB3aXRoIGZvciBsb2dpc3RpYyByZWdyZXNzaW9uIGFuZCAxIGZvciBwb2lzc29uLiBXZSBzdGFydCB3aXRoIHRoZSBsb2dpc3RpYyBvbmVzLiBCZWNhdXNlIHdlIHdpbGwgYmUgdXNpbmcgbXVsdGlwbGUgZGF0YXNldHMgYW5kIHN3aXRjaGluZyBiZXR3ZWVuIHRoZW0sIEkgd2lsbCB1c2UgYXR0YWNoIGFuZCBkZXRhY2ggdG8gdGVsbCBSIHdoaWNoIGRhdGFzZXQgZWFjaCBibG9jayBvZiBjb2RlIHJlZmVycyB0by4gQXR0YWNoIGlzIGEgZnVuY3Rpb24gdGhhdCBicmluZ3MgdGhlIGRhdGFzZXQgaW50byBhY3Rpb24gYW5kIGRldGFjaCBwaGFzZXMgaXQgb3V0IGFnYWluLg0KYGBge3J9DQpjdXNlIDwtIHJlYWQudGFibGUoImh0dHA6Ly9kYXRhLnByaW5jZXRvbi5lZHUvd3dzNTA5L2RhdGFzZXRzL2N1c2UuZGF0IiwgaGVhZGVyID0gVFJVRSkNCm15ZGF0YSA8LSByZWFkLmNzdigiaHR0cDovL3d3dy5hdHMudWNsYS5lZHUvc3RhdC9kYXRhL2JpbmFyeS5jc3YiKQ0KI05vdGljZSB3ZSdyZSBub3Qgc2V0dGluZyBhIHdvcmtpbmcgZGlyZWN0b3J5IGJlY2F1c2Ugd2UncmUgcmVhZGluZyBpbiBkYXRhIGZyb20gdGhlIGludGVybmV0Lg0KYGBgDQojIyBEYXRhIENsZWFuaW5nICYgUmVhZGluZyBPdXRwdXQNCg0KLi4uQmVjYXVzZSBub25lIG9mIHRoaXMgbWF0dGVycyBpZiB5b3VyIGRhdGEgaXMgbm90IGluIGJpbmFyeSBmb3JtISAgICANCg0KTGV0J3Mgc3RhcnQgYnkgbGVhcm5pbmcgc29tZSB3YXlzIHRvIG1hbmFnZSBvdXIgZGF0YSBzbyB3ZSBjYW4gdXNlIGl0IGZvciBhIGRpY2hvdG9tb3VzIG91dGNvbWUuDQoNClRoZSBjdXNlIGRhdGFzZXQgaXMgYW4gb2xkIG9uZSB0aGF0IGluY2x1ZGVzIGFuIE4gPSAxNiB3b21lbiwgYWdlIGdyb3VwIGluIHdoaWNoIHRoZXkgYmVsb25nLCB0aGVpciBlZHVjYXRpb24gbGV2ZWwsIHdoZXRoZXIgdGhleSB3YW50IG1vcmUgY2hpbGRyZW4sIGFuZCB3aGV0aGVyIG9yIG5vdCB0aGV5J3JlIHVzaW5nIGNvbnRyYWNlcHRpdmVzLiBMb2NhdGlvbiBpcyBsaXN0ZWQgYWJvdmUuIEkgdXNlIHRoZSBkYXRhc2V0IGZyb20gUHJpbmNldG9uIGFuZCBtb2RpZnkvYWRkIHRvIHRoZSB0dXRvcmlhbCBieSBHLiBSb2RyaWd1ZXouIEhlcmUncyBhIGxvb2sgYXQgdGhlIGRhdGEgdG8gZ2V0IGEgc2Vuc2Ugb2YgaXQuDQpgYGB7cn0NCmF0dGFjaChjdXNlKQ0Kc3RyKGN1c2UpIA0KbGV2ZWxzKGN1c2UkYWdlKQ0KZGV0YWNoKGN1c2UpDQpgYGANCkhlcmUncyB5b3VyIGJhc2ljIGdsbSBtb2RlbCB3aGVyZWluIHRoZSB3YW50c01vcmVmaXQgKGFzIGluLCB0aGV5IHdhbnQgbW9yZSBjaGlsZHJlbjogeWVzIG9yIG5vKSBvYmplY3QgaXMgb3VyIGZpcnN0IG1vZGVsIHRvIGZpdC4gV2Ugc3BlY2lmeSB0aGUgZ2xtIGZ1bmN0aW9uLCBvdXIgb3V0Y29tZSB2YXJpYWJsZSAoaS5lLiwgd2FudHNNb3JlKSBhbmQgb3VyIHByZWRpY3RvcnMgKGkuZS4sIGFnZSBhbmQgZWR1Y2F0aW9uLCB3aXRob3V0IGFueSBpbnRlcmFjdGlvbiBmYWN0b3JzIHlldCwgaGVuY2Ugd2Ugb25seSB1c2UgKyBmb3IgdGhlIG1haW4gZWZmZWN0cyksIHdlIHRlbGwgaXQgd2hhdCBkYXRhIHRvIHVzZSAoaS5lLiwgY3VzZSksIGFuZCB0aGF0IHRoZSBmYW1pbHkgZnVuY3Rpb24gZm9yIG91ciBkaXN0cmlidXRpb24gKGkuZS4sIGJpbm9taWFsLikgVGhpcyBtb2RlbCBpcyBhc2tpbmcsICJEbyBhZ2UgYW5kIGVkdWNhdGlvbiBwcmVkaWN0IHdhbnRpbmcgbW9yZSBjaGlsZHJlbj8gSWYgeWVzLCB0byB3aGF0IGV4dGVudD8iIEhlcmUsIG91ciBvdXRjb21lICh3YW50aW5nIG1vcmUgY2hpbGRyZW4vd2FudHNNb3JlKSBpcyBhbHJlYWR5IGluIGJpbmFyeSBmb3JtOg0KYGBge3J9DQphdHRhY2goY3VzZSkNCndhbnRzTW9yZWZpdCA8LSBnbG0od2FudHNNb3JlfmFnZStlZHVjYXRpb24sZGF0YT1jdXNlLGZhbWlseT1iaW5vbWlhbCgpKSANCnN1bW1hcnkod2FudHNNb3JlZml0KSAjIGRpc3BsYXkgcmVzdWx0cw0KYGBgDQpTb21lIG5vdGVzIG9uIHRoZSBzdGF0cyB3ZSBnZW5lcmF0ZWQgYWJvdmU6IFVubGlrZSBsaW5lYXIgcmVncmVzc2lvbiwgd2UncmUgdXNpbmcgZ2xtIGFuZCBvdXIgZmFtaWx5IGlzIGJpbm9taWFsLiBJbiBsb2dpc3RpYyByZWdyZXNzaW9uLCB3ZSBhcmUgbm8gbG9uZ2VyIHNwZWFraW5nIGluIHRlcm1zIG9mIGJldGEgc2l6ZXMuIFRoZSBsb2dpc3RpYyBmdW5jdGlvbiBpcyBTLXNoYXBlZCBhbmQgY29uc3RyaWN0cyB0aGUgcmFuZ2UgdG8gMC0xLiBUaHVzLCB3ZSBhcmUgaW5zdGVhZCBjYWxjdWxhdGluZyB0aGUgb2RkcyBvZiBnZXR0aW5nIGEgMCB2cy4gMSBvdXRjb21lLiAgIA0KDQpMZXQncyB3YWxrIHRocm91Z2ggdGhlIG91dHB1dDogVGhlIGZpcnN0IHRoaW5nIHlvdSBzZWUgaXMgdGhlIGRldmlhbmNlIHJlc2lkdWFscywgd2hpY2ggaXMgYSBtZWFzdXJlIG9mIG1vZGVsIGZpdCAoaGlnaGVyIGlzIHdvcnNlLikgTmV4dCBpcyBvdXIgZmlyc3QgYmF0Y2ggb2YgY29lZmZpY2llbnRzLiBHb2luZyBmcm9tIGxlZnQgdG8gcmlnaHQsIHRoZSBDb2VmZmljaWVudHMgdGFibGUgc2hvd3MgeW91IHRoZSAiRXN0aW1hdGUiIGNoYW5nZSBpbiB0aGUgbG9nIG9kZHMgb2YgdGhlIG91dGNvbWUgdmFyaWFibGUgcGVyIG9uZSB1bml0IGluY3JlYXNlIGluIHRoZSBwcmVkaWN0b3IgdmFyaWFibGUsIHN0YW5kYXJkIGVycm9yLCB0aGUgei1zdGF0aXN0aWMgQUtBIHRoZSBXYWxkIHotc3RhdGlzdGljLCBhbmQgdGhlIHAtdmFsdWUuDQoNCkhlcmUncyBhbiBleGFtcGxlIG9mIGhvdyB5b3UnZCBpbnRlcnByZXQgdGhlIGNvZWZmaWNpZW50cyBhYm92ZTogRm9yIGVhY2ggdW5pdCBjaGFuZ2UgaW4gW2luc2VydCBwcmVkaWN0b3IgdmFyaWFibGVdLCB0aGUgbG9nIG9kZHMgb2YgW2FjaGlldmluZyB0aGUgb3V0Y29tZSBvZiBpbnRlcmVzdF0gaW5jcmVhc2VzIGJ5IFtjb2VmZmljaWVudF0uIA0KDQpZb3UndmUgbmV2ZXIgc2VlbiBzb21lb25lIHJlcG9ydCBhbnl0aGluZyBpbiB0ZXJtcyBvZiBsb2cgb2RkcyBhbmQgeW91J3JlIG5vdCBzdXJlIGhvdyB0aGlzIGlzIGEgbWVhbmluZ2Z1bCBkZXNjcmlwdGlvbiBvZiB0aGUgb3V0Y29tZSB2YXJpYWJsZSwgeW91IHNheT8gVGhhdCdzIHdoeSB3ZSBleHBvbmVudGlhdGU6IHRvIHB1dCB0aGlzIGluIHRlcm1zIG9mIG9kZHMgcmF0aW9zIChzZWUgYmVsb3cgZm9yIGludGVycHJldGF0aW9uIG9mIE9Scy4pICANCmBgYHtyfQ0KY29uZmludCh3YW50c01vcmVmaXQpICMgOTUlIENJIGZvciB0aGUgY29lZmZpY2llbnRzDQpgYGANClRoZSBuZXh0IHBhcnQgb2Ygb3VyIG91dHB1dCBpcyB0aGUgOTUlIGNvbmZpZGVuY2UgaW50ZXJ2YWxzIChDSSkgZm9yIHRoZSB1bmV4cG9uZW50aWF0ZWQtY29lZmZpY2llbnRzLiBJbiBsb2dpc3RpYyByZWdyZXNzaW9uLCBpZiB0aGUgY29uZmlkZW5jZSBpbnRlcnZhbCBjcm9zc2VzIG92ZXIgemVybywgYXMgaW4gdGhlIGludGVydmFsIHN0cmV0Y2hlcyBmcm9tIGEgbmVnYXRpdmUgdmFsdWUgdG8gYSBwb3NpdGl2ZSB2YWx1ZSwgdGhhdCBlZmZlY3QgaXMgbm90IHNpZ25pZmljYW50LiANCg0KSGVyZSdzIHRoZSBzYW1lIHRoaW5nIGJlbG93LCBidXQgdGhpcyB0aW1lIHdpdGggZXhwb25lbnRpYXRlZCBjb2VmZmljaWVudHM6DQpgYGB7cn0NCmV4cChjb2VmKHdhbnRzTW9yZWZpdCkpICMgZXhwb25lbnRpYXRlZCBjb2VmZmljaWVudHMNCmV4cChjb25maW50KHdhbnRzTW9yZWZpdCkpICMgOTUlIENJIGZvciBleHBvbmVudGlhdGVkIGNvZWZmaWNpZW50cw0KYGBgDQpJbnRlcnByZXRpbmcgT1JzOiBGb3IgZWFjaCB1bml0IFsiaW5jcmVhc2UiIGlmIE9SIGlzIHBvc2l0aXZlLyJkZWNyZWFzZSIgaWYgT1IgaXMgbmVnYXRpdmVdLCB0aGUgb2RkcyBvZiBbYWNoaWV2aW5nIHRoZSBvdXRjb21lIG9mIGludGVyZXN0IHZzLiBub3RdIGluY3JlYXNlcy9kZWNyZWFzZXMgYnkgYSBmYWN0b3Igb2YgW3RoYXQgcHJlZGljdG9yIHZhcmlhYmxlJ3MgT1JdLiANCmBgYHtyfQ0KcHJlZGljdCh3YW50c01vcmVmaXQsIHR5cGU9InJlc3BvbnNlIikgIyBwcmVkaWN0ZWQgdmFsdWVzDQpyZXNpZHVhbHMod2FudHNNb3JlZml0LCB0eXBlPSJkZXZpYW5jZSIpICMgcmVzaWR1YWxzDQpwbG90KHJlc2lkdWFscyh3YW50c01vcmVmaXQsIHR5cGU9ImRldmlhbmNlIikpICMgcGxvdCB5b3VyIHJlc2lkdWFscw0KZGV0YWNoKGN1c2UpDQpgYGANClJldHVybmluZyB0byBvdXIgYXNzdW1wdGlvbnMsIHdlIGNhbiBub3cgbG9vayBhdCB0aGUgcmVzaWR1Y2FscyBhbmQgcGxvdCB0aGVtIHNvIHdlIGNhbiBzZWUgaWYgdGhleSBsb29rIHdlaXJkLiAoVGhleSBzaG91bGQgbG9vayBsaWtlIGEgYnVuY2ggb2YgcmFuZG9tbHkgc2NhdHRlcmVkIGRvdHMsIG5vdCBhbnkgc3BlY2lhbCBwYXR0ZXJuLikNCg0KQWJvdmUsIG91ciBvdXRjb21lIHZhcmlhYmxlIHdhcyBhbHJlYWR5IGVudGVyZWQgYXMgZWl0aGVyIDAgb3IgMS4gX0J1dCB3aGF0IGlmIHlvdXIgb3V0Y29tZSBpc24ndCBhbHJlYWR5IGluIHN0YW5kYXJkIDAvMSBmb3JtP18gSGVyZSdzIHdoYXQgeW91IGNhbiBkbzogVHJ5IHdvcmtpbmcgYWNyb3NzIGNvbHVtbnMhIElmIHlvdSBjYmluZCB0aGUgdHdvIGNvbHVtbnMgaW50byAxIHZlY3RvciwgUiBjYW4gcmVhZCBvbmUgY29sdW1uIGFzICJoaXRzIiBhbmQgdGhlIG5leHQgYXMgIm1pc3NlcyIganVzdCBsaWtlIHRoZSBwcmV2aW91cyBhbmFseXNlcyByZWFkIHRoZSAwJ3MgYW5kIDEncy4gSGVyZSdzIGhvdyB3ZSBzZXQgdXAgdGhlIGNvbHVtbnMgbGlrZSB0aGF0LiBGWUksIHdlIGNvcHkgYSBuZXcgY29sdW1uIGJlY2F1c2UgSSBwcmVmZXIgbm90IHRvIG92ZXJ3cml0ZSB0aGUgb3JpZ2luYWwgZGF0YSBzbyBJIGNhbid0IHVzZSBpdCBsYXRlci4gQWxzbywgUiByZWFkcyB0aGUgY29sdW1uIGluIG9yZGVyIGFzICJoaXRzIiBhbmQgdGhlbiAibWlzc2VzLCIgc28gaWYgeW91ciBkYXRhIGlzbid0IGFscmVhZHkgaW4gdGhhdCBvcmRlciwgeW91IG1heSBuZWVkIHRvIG1vdmUgb3IgYWRkIGNvbHVtbnMgdG8gZ2V0IHRoZW0gaW4gdGhhdCBvcmRlci4gSGVyZSdzIGhvdy4NCg0KX19Db3B5IGEgQ29sdW1uIGluIFJfXw0KMS4gQ3JlYXRlIHRoZSBuZXcgY29sdW1uDQpgYGB7cn0NCmF0dGFjaChjdXNlKQ0KY3VzZSR3YW50c01vcmVfY29weSA8LSBOQSAjVGhpcyBnaXZlcyB5b3UgYSBjb2x1bW4gb2YgTkFzLiANCmBgYA0KMi4gQ29weSB0aGUgZGF0YSBmcm9tIHRoZSBleGlzdGluZyBjb2x1bW4gaW50byB0aGUgbmV3IG9uZS4NCmBgYHtyfQ0KY3VzZSR3YW50c01vcmVfY29weSA8LSBjdXNlJHdhbnRzTW9yZQ0KDQpjdXNlJHdhbnRzTW9yZV9jb3B5PC1yZWNvZGUoY3VzZSR3YW50c01vcmVfY29weSwiJ25vJz0wO2Vsc2U9MSIpICNOb3cgdGhvc2UgTkFzIGhhdmUgYWN0dWFsIHZhbHVlcy4NCmBgYA0KMy4gUnVuIHRoZSBtb2RlbCBjYmluZGluZyBhY3Jvc3MgdGhlIGNvbHVtbnMgd2hlcmUgdGhlIGZpcnN0IGNvbHVtbiBiZWNvbWVzICJoaXRzIiBhbmQgdGhlIHNlY29uZCBjb2x1bW4gYmVjb21lcyAibWlzc2VzLiIgWW91J2xsIHNlZSBpbiBvdXIgZmlyc3QgbGluZSBvZiB0aGUgbW9kZWwsIHdoZXJlIHRoZSBvdXRjb21lIHZhcmlhYmxlIHVzdWFsbHkgYXBwZWFycywgd2UgaGF2ZSBhIGNiaW5kIGFjcm9zcyBvdXIgdHdvIGNvbHVtbnMgaW5zdGVhZC4gV2UgaGF2ZSBjYmluZGVkIHRoZW0gaW50byBvbmUgdmFyaWFibGUgZm9yIHRoZSBwdXJwb3NlIG9mIHRoaXMgYW5hbHlzaXMuDQpgYGB7cn0NCnVzaW5nZml0IDwtIGdsbShjYmluZCh1c2luZywgbm90VXNpbmcpfmFnZStlZHVjYXRpb24rd2FudHNNb3JlX2NvcHksZGF0YT1jdXNlLGZhbWlseT1iaW5vbWlhbCgpKSANCnN1bW1hcnkodXNpbmdmaXQpICMgZGlzcGxheSByZXN1bHRzDQpjb25maW50KHVzaW5nZml0KSAjIDk1JSBDSSBmb3IgdGhlIGNvZWZmaWNpZW50cw0KZXhwKGNvZWYodXNpbmdmaXQpKSAjIGV4cG9uZW50aWF0ZWQgY29lZmZpY2llbnRzDQpleHAoY29uZmludCh1c2luZ2ZpdCkpICMgOTUlIENJIGZvciBleHBvbmVudGlhdGVkIGNvZWZmaWNpZW50cw0KcHJlZGljdCh1c2luZ2ZpdCwgdHlwZT0icmVzcG9uc2UiKSAjIHByZWRpY3RlZCB2YWx1ZXMNCnJlc2lkdWFscyh1c2luZ2ZpdCwgdHlwZT0iZGV2aWFuY2UiKSAjIHJlc2lkdWFscw0KcGxvdChyZXNpZHVhbHModXNpbmdmaXQsIHR5cGU9ImRldmlhbmNlIikpICMgcGxvdCB5b3VyIHJlc2lkdWFscw0KZGV0YWNoKGN1c2UpDQpgYGANCiMgTG9naXN0aWMgUmVncmVzc2lvbiBBc3N1bXB0aW9ucw0KSGVyZSBhcmUgeW91ciBhc3N1bXB0aW9uczoNCjEuIEluZGVwZW5kZW5jZS4gSnVzdCBrbm93IHlvdXIgZGF0YSwga25vdyB5b3VyIHNhbXBsaW5nIG1ldGhvZHMuIERvIHRoZXkgbWFrZSBhIGRlY2VudCBjYXNlIGZvciBpbmRlcGVuZGVuY2U/IFllcz8gR29kc3BlZWQsIGNsYXNzbWF0ZXMuIE9udG8gYXNzdW1wdGlvbiAjMi4NCjIuIE5vbi1maXNoeSBkaXN0cmlidXRpb24gb2YgdGhlIHJlc2lkdWFscy4gKFNlZSByZXNpZHVhbHMgcGxvdCBhYm92ZS4pDQozLiBDb3JyZWN0IHNwZWNpZmljYXRpb24gb2YgdGhlIHZhcmlhbmNlIHN0cnVjdHVyZQ0KNC4gTGluZWFyIHJlbGF0aW9uc2hpcCBiZXR3ZWVuIHRoZSByZXNwb25zZSBhbmQgdGhlIGxpbmVhciBwcmVkaWN0b3IgIA0KRm9yIDItNCwgeW91IG5lZWQgdG8gdXNlIHlvdXIgbW9kZWwuIEFsc28sIHlvdXIgcHJlZGljdG9yIG1heSBub3QgYmUgbGluZWFyLCBzbyBkb24ndCBiZSBhIHBlcmZlY3Rpb25pc3QuIExldCdzIGdldCBpbnRvIHRoZSBhbmFseXNpcywgdGhlbi4gIA0KDQpJbiAqbXlkYXRhKiwgb3VyIG91dGNvbWUgdmFyaWFibGUgaXMgYWRtaXQgKGkuZS4sIGFkbWlzc2lvbiB0byBncmFkIHNjaG9vbCwgd2hpY2ggaXMgYSB5ZXMvbm8gcmVzcG9uc2UuKSBXZSd2ZSBhbHNvIGdvdCBwcmVkaWN0b3IgdmFyaWFibGVzOiBncmUgKGkuZS4sIEdSRSBleGFtIHNjb3JlKSBhbmQgZ3BhICh1bmRlcmdyYWR1YXRlIEdQQSksIHdoaWNoIGFyZSBjb250aW51b3VzKGlzaCksIGFuZCByYW5rIChyYW5raW5nIG9mIHVuZGVyZ3JhZCBpbnN0aXR1dGlvbiB3aGVyZSAxIGlzIHRoZSBiZXN0IGFuZCA0IGlzIHRoZSB3b3JzdCwgc29ydCBvZiBsaWtlIHRpZXJzKSB3aGljaCBpcyBtb3JlIG9yZGluYWwvY2F0ZWdvcmljYWwgYXMgYSB2YXJpYWJsZS4gVGhpcyBkYXRhc2V0IGlzIGFza2luZyB3aGV0aGVyIGFuZCB0byB3aGF0IGV4dGVudCBHUEEsIEdSRSBzY29yZSwgYW5kIHJhbmsgb2YgaW5zdGl0dXRpb24gY2hhbmdlcyB0aGUgbGlrZWxpaG9vZCBvZiBnZXR0aW5nIGFkbWl0dGVkIHRvIGdyYWQgc2Nob29sLiAgIA0KDQpMZXQncyB0YWtlIGEgbG9vayBhdCBvdXIgZGF0YSBhbmQgbWFrZSBzdXJlIHdlIGluIGZhY3QgaGF2ZSBhIGJpbmFyeSBvdXRjb21lLiBDZXJ0YWluIGdyYXBocyBhbmQgYW5hbHlzZXMgcmVxdWlyZSBjb250aW51b3VzIHZzLiBjYXRlZ29yaWNhbCBwcmVkaWN0b3JzLCBzbyB3ZSBjYW4gY29uZmlybSB3aGF0IHdlJ3JlIHdvcmtpbmcgd2l0aCBieSBnZXR0aW5nIGEgc3VtbWFyeS4gS25vd2luZyBvdXIgZGF0YSBzaG91bGQgaGVscCB1cyBjb3ZlciB0aGUgMXN0IGFzc3VtcHRpb24uIFRoaXMgd2lsbCB1c2UgdGhlIGRhdGEgZnJvbSBhbmQgbW9kaWZ5L2FkZCBvbnRvIHRoZSBVQ0xBIFN0YXRpc3RpY2FsIENvbnN1bHRpbmcgR3JvdXAncyB0dXRvcmlhbC4gVGhlaXIgZGF0YXNldCBpcyBkaWZmZXJlbnQgZnJvbSBjdXNlIGFuZCBhbGxvd3MgdXMgdG8gZG8gZGlmZmVyZW50IGxvZ2lzdGljIGFuYWx5c2VzLiAgDQpgYGB7ciBlY2hvPUZBTFNFfQ0KYXR0YWNoKG15ZGF0YSkNCnN1bW1hcnkobXlkYXRhKQ0KI0J1dCB0aGUgc3VtbWFyeSBkb2Vzbid0IGdpdmUgdXMgdGhlIHN0YW5kYXJkIGRldmlhdGlvbiwgc28gd2UgbmVlZCB0byBhZGQgdGhhdCBvdXJzZWx2ZXMuDQpzYXBwbHkobXlkYXRhLCBzZCkNCg0KYGBgDQoNCkhlcmUgYXJlIHNvbWUgd2F5cyB0byBleWViYWxsIHRoZSBkYXRhIGZvciB5b3VyIGFzc3VtcHRpb25zLiBJcyB5b3VyIG91dGNvbWUgdmFyaWFibGUgZGljaG90b21vdXM/IFRoZXJlIHNob3VsZCBvbmx5IGJlIDIgcG9zc2libGUgb3V0Y29tZXMgKG9yIGxldmVscykgZm9yIHRoYXQgdmFyaWFibGUuIFZpc3VhbGl6aW5nIHlvdXIgZGF0YSBpbiBhIHRhYmxlIHdpbGwgc2hvdyB5b3UgaG93IG1hbnkgbGV2ZWxzIHlvdSdyZSBkZWFsaW5nIHdpdGguIFRha2UgYSBsb29rOiBTaW5jZSBhZG1pdCAoYWRtaXNzaW9uIHRvIGdyYWQgc2Nob29sLCB5ZXMgb3Igbm8pIGlzIG91ciBvdXRjb21lIHZhcmlhYmxlIGhlcmUsIGRvZXMgaXQgc2hvdyB5b3UgZXhhY3RseSAyIGxldmVscyBvZiBhZG1pdCBpbiB0aGUgdGFibGU/IElmIHllcywgdGhhdCdzIGEgcG9pbnQgdG93YXJkIG1lZXRpbmcgb3VyIGFzc3VtcHRpb25zLg0KYGBge3J9DQoNCnh0YWJzKH4gYWRtaXQgKyByYW5rLCBkYXRhID0gbXlkYXRhKSANCg0KYGBgDQoNCkEgY29uZGl0aW9uYWwgZGVuc2l0eSBwbG90IHdpbGwgYWxzbyBzaG93IHVzIGlmIG91ciBkaXN0cmlidXRpb24gbG9va3MgYmluYXJ5LiBZb3UncmUgbG9va2luZyBmb3IgYSBkaWFnb25hbChpc2gpIHBsb3Qgd2l0aCBvZGRzIHRoYXQgZGlmZmVyIGFjY29yZGluZyB0byB0aGUgbGV2ZWwgKDAgb3IgMSkgb2YgeW91ciBvdXRjb21lIHZhcmlhYmxlLg0KDQpXaGF0IGlmIGl0IGlzbid0IHJlYWRpbmcgdGhlIERWIGFzIGEgZmFjdG9yPyBFYXJsaWVyIGl0IGdhdmUgdXMgYSBtZWFuIG9uIHRoaXMsIHNvIGl0J3MgcHJvYmFibHkgcmVhZGluZyBpdCBhcyBhIG51bWVyaWMgdmFsdWUsIG5vdCBhcyBhIGZhY3RvciwgbGlrZSBpdCBuZWVkcyB0byBiZS4gTGV0J3MgbWFrZSBhIDJuZCB2ZXJzaW9uIG9mIGFkbWl0IHRoYXQgUiB3aWxsIGRlZmluaXRlbHkgcmVhZCBhcyBhIGZhY3RvciB3aXRoIDIgbGV2ZWxzLiBLZWVwIHRoaXMgaW4gbWluZDogaXQncyBhIHZlcnkgY29tbW9uIGVycm9yIGluIGxvZ2lzdGljIHJlZ3Jlc3Npb24gaW4gUi4gSGVyZSdzIGhvdyB0byByZW1pbmQgUiB0aGF0IGl0J3MgZGVmaW5pdGVseSBhIGZhY3RvcjoNCmBgYHtyfQ0KDQpteWRhdGEkYWRtaXQyID0gZmFjdG9yKG15ZGF0YSRhZG1pdCkNCg0KYGBgDQoNCldlIHRlbGwgUiB0byBhZGQgYSB2YXJpYWJsZSBjYWxsZWQgYWRtaXQyIHRvIG15ZGF0YSwgd2hpY2ggaXMgYSBmYWN0b3JlZCB2ZXJzaW9uIG9mIG91ciBvcmlnaW5hbCBhZG1pdC4NCmBgYHtyfQ0KbGV2ZWxzKG15ZGF0YSRhZG1pdDIpID0gYygibm8iLCJ5ZXMiKSANCmBgYA0KDQpBbmQganVzdCB0byBtYWtlIHN1cmUgaXQncyBub3QgcmVhZCBhcyBhIG51bWVyaWMgYWdhaW4sIGxldCdzIHB1dCBzb21lIG5vbi1udW1lcmljIGxldmVscyBvbiBhZG1pdDIuIElmIHlvdSBjbGljayBvbiBteWRhdGEgaW4gdGhlIEdsb2JhbCBFbnZpcm9ubWVudCwgeW91IHNob3VsZCBzZWUgYWRtaXQyIGFzIGEgY29sdW1uIHdpdGggeWVzIGFuZCBubyBjb3JyZXNwb25kaW5nIHRvIHRoZSAwJ3MgYW5kIDEncyBpbiBhZG1pdC4gDQoNClJlbWluZGVyOiAwID0gbm8sIDEgPSB5ZXMuDQoNCk5vdyBsZXQncyB0cnkgdGhhdCBjb25kaXRpb25hbCBkZW5zaXR5IHBsb3Qgd2l0aCBhZG1pdDIuDQpgYGB7ciBlY2hvPUZBTFNFfQ0KY2RwbG90KG15ZGF0YSRncmUsIG15ZGF0YSRhZG1pdDIsDQogICAgICAgcGxvdCA9IFRSVUUsIHRvbC55bGFiID0gMC4wNSwgeWxldmVscyA9IE5VTEwsDQogICAgICAgYncgPSAibnJkMCIsIG4gPSA0MDAsIGZyb20gPSBOVUxMLCB0byA9IE5VTEwsDQogICAgICAgY29sID0gTlVMTCwgYm9yZGVyID0gMSwgbWFpbiA9ICIiLCB4bGFiID0gTlVMTCwgeWxhYiA9IE5VTEwsDQogICAgICAgeWF4bGFiZWxzID0gTlVMTCwgeGxpbSA9IE5VTEwsIHlsaW0gPSBjKDAsIDEpKQ0KDQpgYGANCg0KIyMgTG9naXN0aWMgUmVncmVzc2lvbiBhbmQgQ2F0ZWdvcmljYWwgUHJlZGljdG9yIFZhcmlhYmxlcyB0aGF0IGFyZSBBbHNvIE9yZGluYWwgIA0KDQpBYm92ZSB3ZSByYW4gYSBiYXNpYyBnbG0gbW9kZWwgYW5kIGxvb2tlZCBhdCBtYWluIGVmZmVjdHMuIEhlcmUncyBob3cgd2UgY2FuIGRvIHNvbWUgb3RoZXIgYW5hbHlzZXMgd2l0aCBvdXIgZ3JhZCBzY2hvb2wgYWRtaXQgZGF0YS4gSGVyZSdzIGEgY29vbCBjYXRlZ29yaWNhbCBhbmFseXNpcyBmcm9tIHRoZSBVQ0xBIExvZ2l0IFJlZ3Jlc3Npb24gVHV0b3JpYWwgdGhhdCBJIHRoaW5rIGlzIHdvcnRoIGtub3dpbmcgeW91IGNhbiBkbyB3aXRoIGEgbG9naXN0aWMgbW9kZWwuIEhlcmUsIHJhbmsgaGFzIGFuIG9yZGVyIGFuZCB3ZSBjYW4gYW5hbHl6ZSB0aGUgcG90ZW50aWFsIGNoYW5nZSBpbiB0aGUgb3V0Y29tZSAoYWRtaXNzaW9uIHRvIGdyYWQgc2Nob29sKSBkZXBlbmRpbmcgb24gZWFjaCBsZXZlbCBvZiByYW5rLg0KDQpgYGB7ciBlY2hvPUZBTFNFfQ0KDQpteWRhdGEkcmFuayA8LSBmYWN0b3IobXlkYXRhJHJhbmspDQpteWxvZ2l0IDwtIGdsbShhZG1pdCB+IGdyZSArIGdwYSArIHJhbmssIGRhdGEgPSBteWRhdGEsIGZhbWlseSA9ICJiaW5vbWlhbCIpDQpzdW1tYXJ5KG15bG9naXQpDQpjb25maW50KG15bG9naXQpICMjIENJcyB1c2luZyBwcm9maWxlZCBsb2ctbGlrZWxpaG9vZA0KY29uZmludC5kZWZhdWx0KG15bG9naXQpICMjIENJcyB1c2luZyBzdGFuZGFyZCBlcnJvcnMNCg0KI1Rlcm1zID0gNDo2IHJlZmVycyB0byBSYW5rcyAyLCAzLCAmIDQgYmVpbmcgdGhlIDR0aCB0aHJvdWdoIHRoZSA2dGggdGVybXMgb2YgdGhlIG1vZGVsDQp3YWxkLnRlc3QoYiA9IGNvZWYobXlsb2dpdCksIFNpZ21hID0gdmNvdihteWxvZ2l0KSwgVGVybXMgPSA0OjYpDQoNCmBgYA0KDQpUaGUgcC12YWx1ZSBpbiB0aGUgYWJvdmUgd2FsZCB0ZXN0IHRlbGxzIHlvdSBpZiB0aGUgb3ZlcmFsbCBlZmZlY3Qgb2YgdGhhdCBjYXRlZ29yaWNhbCB2YXJpYWJsZSBpcyBzaWduaWZpY2FudC4gIA0KDQpUaGlzIGlzIGFsbW9zdCBha2luIHRvIHJ1bm5pbmcgYW4gb21uaWJ1cyB0ZXN0IGluIEFOT1ZBLiBXZSBrbm93IHNvbWV0aGluZyBpcyBoYXBwZW5pbmcgd2l0aCByYW5rLCBzbyBoZXJlJ3MgaG93IHlvdSBjYW4gY29tcGFyZSBsZXZlbHMgb2YgcmFuay4NCmBgYHtyIGVjaG89RkFMU0V9DQojVGVybXMgeW91J3JlIGNvbXBhcmluZyBvciBub3QgY29tcGFyaW5nIG5lZWQgdG8gc3RheSBpbiB0aGUgb3JkZXIgaW4gd2hpY2ggdGhleSBhcHBlYXIgaW4gdGhlIG1vZGVsDQpjb21wMSA8LSBjYmluZCgwLDAsMCwxLC0xLDApIA0Kd2FsZC50ZXN0KGIgPSBjb2VmKG15bG9naXQpLCBTaWdtYSA9IHZjb3YobXlsb2dpdCksIEwgPSBjb21wMSkgDQoNCiMjIG9kZHMgcmF0aW9zIG9ubHkNCmV4cChjb2VmKG15bG9naXQpKQ0KDQojIyBvZGRzIHJhdGlvcyBhbmQgOTUlIENJDQpleHAoY2JpbmQoT1IgPSBjb2VmKG15bG9naXQpLCBjb25maW50KG15bG9naXQpKSkgDQpkZXRhY2gobXlkYXRhKQ0KYGBgDQojIyBIZXJlIEFyZSB0aGUgUmVzdCBvZiB5b3VyIFN0YXRzIHRvIExvb2sgYXQgSW50ZXJhY3Rpb25zDQpgYGB7cn0NCmF0dGFjaChjdXNlKQ0KSW50dXNpbmdmaXQgPC0gZ2xtKGNiaW5kKHVzaW5nLCBub3RVc2luZyl+YWdlKmVkdWNhdGlvbixkYXRhPWN1c2UsZmFtaWx5PWJpbm9taWFsKCkpIA0Kc3VtbWFyeShJbnR1c2luZ2ZpdCkgIyBkaXNwbGF5IHJlc3VsdHMNCmNvbmZpbnQoSW50dXNpbmdmaXQpICMgOTUlIENJIGZvciB0aGUgY29lZmZpY2llbnRzDQpleHAoY29lZihJbnR1c2luZ2ZpdCkpICMgZXhwb25lbnRpYXRlZCBjb2VmZmljaWVudHMNCmV4cChjb25maW50KEludHVzaW5nZml0KSkgIyA5NSUgQ0kgZm9yIGV4cG9uZW50aWF0ZWQgY29lZmZpY2llbnRzDQpwcmVkaWN0KEludHVzaW5nZml0LCB0eXBlPSJyZXNwb25zZSIpICMgcHJlZGljdGVkIHZhbHVlcw0KcmVzaWR1YWxzKEludHVzaW5nZml0LCB0eXBlPSJkZXZpYW5jZSIpICMgcmVzaWR1YWxzDQoNCjEtcGNoaXNxKDczLjAzMyw4KSAjIDEgLSBjaGlzcXVhcmUgKHJlc2lkdWFsIGRldmlhbmNlLCBERikNCg0KI0FuYWx5c2lzIG9mIERldmlhbmNlIFRhYmxlDQphbm92YShJbnR1c2luZ2ZpdCwgdGVzdD0iQ2hpc3EiKSANCg0KI01jRmFkZGVuIFJeMiBpcyBhIHBzZXVkbyBSXjIgZm9yIGxvZ2lzdGljIHJlZ3Jlc3Npb24NCnBSMihJbnR1c2luZ2ZpdCkNCmBgYA0KDQojIyBVcGRhdGluZyB0aGUgTW9kZWwNCg0KV2hhdCBpZiB5b3UgKG9yIHlvdXIgUEkpIHN1ZGRlbmx5IGRlY2lkZSB5b3Ugd2FudCB0byBzZWUgdGhlIGFuYWx5c2lzLCBidXQgV0lUSE9VVCB0aGF0IG9uZSBwcmVkaWN0b3IgdmFyaWFibGU/IEZyZXQgbm90LCB5b3UgZG9uJ3QgbmVlZCB0byByZXdyaXRlIHRoZSB3aG9sZSB0aGluZy4gWW91IGNhbiB1c2UgdGhlIHVwZGF0ZSBmdW5jdGlvbiBpbnN0ZWFkLiBTaW5jZSBpdCdzIGVhc2llciB0byBkZW1vbnN0cmF0ZSB0aGlzIHdpdGggYSBzaW1wbGUgYWRkaXRpdmUgbW9kZWwsIHdlJ2xsIGxvb2sgYXQgaXQgd2l0aCB0aGUgb2xkIHVzaW5nZml0IG9uZSwgd2hpY2ggb25seSBsb29rZWQgYXQgbWFpbiBlZmZlY3RzIGFuZCBub3QgaW50ZXJhY3Rpb25zLiBUaGlzIGlzIHRoZSBtb2RlbCBmcm9tIHRoZSAzcmQgc2VjdGlvbiBvZiAiQ29weWluZyBhIENvbHVtbiBpbiBSLiIgTm93IHdlJ2xsIGxvb2sgYXQgaXQgd2l0aG91dCBlZHVjYXRpb24uDQpgYGB7cn0NClVwZGF0ZXVzaW5nZml0IDwtIHVwZGF0ZSh1c2luZ2ZpdCwgfiAuIC0gZWR1Y2F0aW9uKSANCnN1bW1hcnkoVXBkYXRldXNpbmdmaXQpDQpkZXRhY2goY3VzZSkNCmBgYA0KUmVnYXJkaW5nIHRoZSBNY0ZhZGRlbiBSXjIsIHdoaWNoIGlzIGEgcHNldWRvIFJeMiBmb3IgbG9naXN0aWMgcmVncmVzc2lvbi4uLkEgcmVndWxhciAoaS5lLiwgbm9uLXBzZXVkbykgUl4yIGluIG9yZGluYXJ5IGxlYXN0IHNxdWFyZXMgcmVncmVzc2lvbiBpcyBvZnRlbiB1c2VkIGFzIGFuIGluZGljYXRvciBvZiBnb29kbmVzcy1vZi1maXQuIEhvd2V2ZXIsIGluIGEgbG9naXN0aWMgcmVncmVzc2lvbiB3ZSBkb24ndCBoYXZlIHRoZSB0eXBlcyBvZiB2YWx1ZXMgdG8gY2FsY3VsYXRlIGEgcmVhbCBSXjIuIEEgcHNldWRvIFJeMiBoZXJlIGlzIGNhbGN1bGF0ZWQgZGlmZmVyZW50bHkgYW5kIGlzIHNpbWlsYXJseSB1c2VkIGFzIGEgbWVhc3VyZSBvZiBnb29kbmVzcyBvZiBmaXQgKGxvd2VyIGlzIGJldHRlci4pIEJ1dCB0aGF0J3Mgbm90IGFsbCB0aGVyZSBpcyB0byBwc2V1ZG8tUl4yJ3MuIFtIZXJlJ3NdKGh0dHA6Ly93d3cuYXRzLnVjbGEuZWR1L3N0YXQvbXVsdF9wa2cvZmFxL2dlbmVyYWwvUHN1ZWRvX1JTcXVhcmVkcy5odG0pIGEgZ29vZCBleHBsYW5hdGlvbiB3aXRoIG1vcmUgZGV0YWlsLiANCg0KIyMgR3JhcGhpbmcgTG9naXN0aWMgUmVncmVzc2lvbiBBbmFseXNlcyAgDQoNClRoaXMgdGFrZXMgc29tZSB3b3JrLiBCZWNhdXNlIHdlIHRhbGsgYWJvdXQgYmluYXJ5IG91dGNvbWVzIGluIHRlcm1zIG9mIG9kZHMgcmF0aW9zLCBidXQgaXQncyBub3QgdmVyeSBpbmZvcm1hdGl2ZSB0byBncmFwaCBvZGRzIHJhdGlvcywgd2UgbmVlZCB0byB3b3JrIGluc3RlYWQgd2l0aCBwcmVkaWN0ZWQgcHJvYmFiaWxpdGllcyBhbmQgZ3JhcGggdGhvc2UuIFRoYXQgaW52b2x2ZXMgbWFraW5nIGEgYnVuY2ggb2YgbmV3IGRhdGFzZXRzLiBUaGlzIGlzIG15IGFubm90YXRlZCB2ZXJzaW9uIG9mIHRoZSBVQ0xBIHR1dG9yaWFsIGZvciBiZWdpbm5lcnMsIGFsb25nIHdpdGggdGl0bGluZyB0aGUgZ3JhcGggYmVjYXVzZSB1bnRpdGxlZCBncmFwaHMgYXJlIG15IHBldCBwZWV2ZS4NCg0KMS4gU3RhcnQgYnkgY2FsY3VsYXRpbmcgdGhlIHByZWRpY3RlZCBwcm9iYWJpbGl0eSBvZiB0aGUgb3V0Y29tZSBhdCBlYWNoIGxldmVsIG9mIHRoZSBjYXRlZ29yaWNhbC9vcmRpbmFsIHByZWRpY3RvciB2YXJpYWJsZSBhbmQvb3IgaG9sZGluZyBjb250aW51b3VzIHZhcmlhYmxlcyBhdCB0aGVpciBtZWFucy4gDQpgYGB7cn0NCmF0dGFjaChteWRhdGEpDQpuZXdkYXRhMSA8LSB3aXRoKG15ZGF0YSwNCiAgICAgICAgICAgICAgICAgZGF0YS5mcmFtZShncmUgPSBtZWFuKGdyZSksIGdwYSA9IG1lYW4oZ3BhKSwgcmFuayA9IGZhY3RvcigxOjQpKSkNCmBgYA0KMi4gQ3JlYXRlIHRoZSBwcmVkaWN0ZWQgcHJvYmFiaWxpdGllcyB1c2luZyB0aGUgbmV3IGRhdGFmcmFtZQ0KYGBge3J9DQpuZXdkYXRhMSRyYW5rUCA8LSBwcmVkaWN0KG15bG9naXQsIG5ld2RhdGEgPSBuZXdkYXRhMSwgdHlwZSA9ICJyZXNwb25zZSIpDQpuZXdkYXRhMQ0KYGBgDQogX05vdGU6XyBUaGlzIGFjdHVhbGx5IHJlcXVpcmVzIHVzaW5nIHJlcCBhbmQgc2VxIGZ1bmN0aW9ucyBpbiBvcmRlciB0byBtYWtlIG91ciBkYXRhIGZpdCBvdXIgR1JFIHNjb3JlIHJhbmdlIGFuZCBoYXZlIHRpYyBtYXJrcyBhdCAxMDAgcG9pbnRzLCBzaW1pbGFyIHRvIHdoYXQgb3VyIFNEIHNob3dlZCB1cyBhYm91dCB0aGUgR1JFIHNjb3JlcycgZGlzdHJpYnV0aW9uLiBXZSBhbHNvIGhhdmUgdG8gY29uc3RhbnRseSByZW1pbmQgUiB0aGF0IG91ciBmYWN0b3JzIGFyZSBpbmRlZWQgZmFjdG9ycy4gSWYgeW91IGhhdmUgc29tZXRoaW5nIGxvZ2lzdGljIHRoYXQgd29uJ3QgcnVuLCB0aGlzIGlzIGEgY29tbW9uIHJlYXNvbiB3aHkuICANCg0KMy4gU3RhcnQgY3JlYXRpbmcgdGhlIHByZWRpY3RlZCBwcm9iYWJpbGl0eSB2ZXJzaW9uIG9mIHlvdXIgdmFyaWFibGVzDQpgYGB7cn0NCm5ld2RhdGEyIDwtIHdpdGgobXlkYXRhLA0KICAgICAgICAgICAgICAgICBkYXRhLmZyYW1lKGdyZSA9IHJlcChzZXEoZnJvbSA9IDIwMCwgdG8gPSA4MDAsIGxlbmd0aC5vdXQgPSAxMDApLCA0KSwNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICBncGEgPSBtZWFuKGdwYSksIHJhbmsgPSBmYWN0b3IocmVwKDE6NCwgZWFjaCA9IDEwMCkpKSkNCmBgYA0KNC4gU2V0IHRoZSBTRSB0byBkaXNwbGF5DQpgYGB7cn0NCm5ld2RhdGEzIDwtIGNiaW5kKG5ld2RhdGEyLCBwcmVkaWN0KG15bG9naXQsIG5ld2RhdGEgPSBuZXdkYXRhMiwgdHlwZT0ibGluayIsIHNlPVRSVUUpKQ0KbmV3ZGF0YTMgPC0gd2l0aGluKG5ld2RhdGEzLCB7DQogIFByZWRpY3RlZFByb2IgPC0gcGxvZ2lzKGZpdCkNCiAgTEwgPC0gcGxvZ2lzKGZpdCAtICgxLjk2ICogc2UuZml0KSkNCiAgVUwgPC0gcGxvZ2lzKGZpdCArICgxLjk2ICogc2UuZml0KSkNCn0pDQpgYGANCjUuIEZpbmFsbHksIHRoZSBncmFwaA0KYGBge3J9DQpQcmVkUHJvYlBsb3QgPC0gZ2dwbG90KG5ld2RhdGEzLCBhZXMoeCA9IGdyZSwgeSA9IFByZWRpY3RlZFByb2IpKSArDQogIGdlb21fcmliYm9uKGFlcyh5bWluID0gTEwsIHltYXggPSBVTCwgZmlsbCA9IHJhbmspLCBhbHBoYSA9IC4yKSArDQogIGdlb21fbGluZShhZXMoY29sb3VyID0gcmFuayksIHNpemU9MSkNCg0KUHJlZFByb2JQbG90ICsgZ2d0aXRsZSgiQWRtaXNzaW9uIHRvIEdyYWQgU2Nob29sIGJ5IFJhbmsiKSAjYWRkIGEgdGl0bGUNCmRldGFjaChteWRhdGEpDQpgYGANCkludGVycHJldGluZyB0aGUgZ3JhcGg6IEVhY2ggbGluZSBpcyB0aGUgcHJlZGljdGVkIHByb2JhYmlsaXR5IG9mIGJlaW5nIGFkbWl0dGVkIHRvIGdyYWQgc2Nob29sIGZvciBlYWNoIGluc3RpdHV0aW9uYWwgcmFuay4gVGhlIGxlZ2VuZCB0ZWxscyB1cyB0aGF0IHJlZCByZXByZXNlbnRzICMxIHJhbmtlZCBpbnN0aXR1dGlvbnMsIGdyZWVuIHJlcHJlc2VudHMgIzIgcmFua2VkIGluc3RpdHV0aW9ucywgYmx1ZSByZXByZXNlbnRzICMzIHJhbmtlZCBpbnN0aXR1dGlvbnMsIGFuZCBwdXJwbGUgcmVwcmVzZW50cyAjNCByYW5rZWQgaW5zdGl0dXRpb25zIChpbiBvcmRlciBmcm9tIHRvcCByYW5rZWQgdG8gbG93ZXN0IHJhbmtlZC4pIEVhY2ggaXMgc2V0IGFnYWlzdCB0aGUgY29sb3ItY29kZWQgY29uZmlkZW5jZSBpbnRlcnZhbC4gV2UgY2FuIHNlZSB0aGF0IHRoZSBpbnRlcmNlcHRzIGFsbCBmYWxsIGluIHJhbmsgb3JkZXIgYW5kIHRoYXQgZm9yIGVhY2ggaW5zdGl0dXRpb24gcmFuaywgdGhlIHBvc2l0aXZlIHNsb3BlIHNob3dzIHRoZSBwcmVkaWN0ZWQgcHJvYmFiaWxpdHkgb2YgYmVpbmcgYWRtaXR0ZWQgdG8gZ3JhZCBzY2hvb2wgaW5jcmVhc2VzIGFzIEdSRSBzY29yZSBpbmNyZWFzZXMuICANCg0KUGxvdHRpbmcgYW4gSW50ZXJhY3Rpb24gVXNpbmcgdGhlIEVmZmVjdHMgUGFja2FnZQ0KYGBge3J9DQphdHRhY2goY3VzZSkNCkFnZVdhbnRmaXQgPC0gZ2xtKGNiaW5kKHVzaW5nLG5vdFVzaW5nKX4NCiAgICAgICAgICAgICAgICAgICAgYWdlKndhbnRzTW9yZV9jb3B5LGRhdGE9Y3VzZSwNCiAgICAgICAgICAgICAgICAgIGZhbWlseT1iaW5vbWlhbCgpKQ0Kc3VtbWFyeShBZ2VXYW50Zml0KSAjIGRpc3BsYXkgcmVzdWx0cw0KDQpwbG90KGVmZmVjdCgiYWdlOndhbnRzTW9yZV9jb3B5IiwNCiAgICAgICAgICAgIEFnZVdhbnRmaXQsDQogICAgICAgICAgICB4bGV2ZWxzID0gbGlzdChhZ2U9NCwgd2FudHNNb3JlX2NvcHk9YygxLDApKSksDQogICAgIHJ1Zz1GLA0KICAgICBtdWx0aWxpbmU9VCwNCiAgICAgbWFpbj0iQWdlIGJ5IFdhbnRzIE1vcmUgSW50ZXJhY3Rpb24gXG4gdG8gUHJlZGljdCBVc2luZyBDb250cmFjZXB0aXZlcyIsDQogICAgIHlsYWI9IkNvbnRyYWNlcHRpdmUgVXNlIiwNCiAgICAgeGxhYj0iQWdlIikNCmRldGFjaChjdXNlKQ0KYGBgDQpJbnRlcnByZXRpbmcgdGhlIGdyYXBoOiBUaGUgbGVnZW5kIHRlbGxzIHVzIHRoYXQgdGhlIGJsYWNrIGxpbmUgd2l0aCBjaXJjbGVzIHJlcHJlc2VudHMgd29tZW4gd2hvIERPTidUIHdhbnQgbW9yZSBraWRzIGFuZCB0aGUgcmVkIGRhc2hlZCBsaW5lIHdpdGggdHJpYW5nbGVzIHJlcHJlc2VudHMgd29tZW4gd2hvIERPIHdhbnQgbW9yZSBraWRzLiBXZSBzZWUgdGhhdCwgaW4gZ2VuZXJhbCBhY3Jvc3MgYm90aCBncm91cHMsIHRoZSBvbGRlciB3b21lbiBhcmUsIHRoZSBtb3JlIGxpa2VseSB0aGV5IGFyZSB0byB1c2UgY29udHJhY2VwdGl2ZXMuIEhvd2V2ZXIsIHdlIHNlZSBhbiBHcm91cCBieSBBZ2UgaW50ZXJhY3Rpb24uIFRoZSBzbG9wZSBpcyBtdWNoIHN0ZWVwZXIgZm9yIHdvbWVuIHdobyBET04nVCB3YW50IG1vcmUga2lkczogdGhleSBhcmUgbWFya2VkbHkgbW9yZSBsaWtlbHkgdG8gdXNlIGNvbnRyYWNlcHRpdmVzIHRoYW4gd29tZW4gd2hvIERPIHdhbnQgbW9yZSBraWRzIGFib3ZlLiBXb21lbiB3aG8gRE8gd2FudCBtb3JlIGtpZHMgZXhwZXJpZW5jZSBsZXNzIG9mIGFuIGltcGFjdCBvZiBhZ2Ugb24gdGhlaXIgY29udHJhY2VwdGl2ZSB1c2UgdGhhbiBkbyB3b21lbiB3aG8gRE9OJ1Qgd2FudCBtb3JlIGtpZHMuDQoNCiMgUG9pc3NvbiBSZWdyZXNzaW9uIA0KIyMgVGhhdCwgYnV0IENoYW5nZSB0aGUgZmFtaWx5IHRvICJQb2lzc29uIg0KDQpObywgYnV0IHNlcmlvdXNseSwgaGVyZSdzIHRoZSBlbnRpcmUgUG9pc3NvbiBzZWN0aW9uIG9uIFJvYmVydCBJLiBLYWJhY29mZidzIHF1aWNrUiBibG9nIGF0IGh0dHA6Ly93d3cuc3RhdG1ldGhvZHMubmV0L2FkdnN0YXRzL2dsbS5odG1sOg0KYGBge3J9DQojUG9pc3NvbiBSZWdyZXNzaW9uDQojd2hlcmUgY291bnQgaXMgYSBjb3VudCBhbmQgDQojIHgxLXgzIGFyZSBjb250aW51b3VzIHByZWRpY3RvcnMgDQojZml0IDwtIGdsbShjb3VudCB+IHgxK3gyK3gzLCBkYXRhPW15ZGF0YSwgZmFtaWx5PXBvaXNzb24oKSkNCiNzdW1tYXJ5KGZpdCkgZGlzcGxheSByZXN1bHRzDQpgYGANClNlZT8gSSB3YXNuJ3Qgam9raW5nLiBCdXQgbGV0J3MgZG8gb25lIGZvciByZWFsLiBIZXJlJ3MgZGF0YSBJIGdvdCBmcm9tIGh0dHA6Ly93d3cudGhlYW5hbHlzaXNmYWN0b3IuY29tL2dlbmVyYWxpemVkLWxpbmVhci1tb2RlbHMtaW4tci1wYXJ0LTYtcG9pc3Nvbi1yZWdyZXNzaW9uLWNvdW50LXZhcmlhYmxlcy8gYnkgRGF2aWQgTGlsbGlzLg0KDQpJdCdzIGFib3V0IGNhc2VzIChpLmUuLCBjb3VudHMpIG9mIGRpc2Vhc2UgYW1vbmcgaGlnaCBzY2hvb2wgc3R1ZGVudHMgYnkgbnVtYmVyIG9mIGRheXMgYWZ0ZXIgb3V0YnJlYWsuIEhlcmUncyB0aGUgZGF0YSwgY2FsbGVkICdjYXNlcy4nIEVhY2ggdGltZSwgcnVuIHRoZSB3aG9sZSBjaHVuayBhdCBvbmNlIG9yIGl0IHdvbid0IHdvcmsuDQpgYGB7cn0NCmNhc2VzIDwtICANCnN0cnVjdHVyZShsaXN0KERheXMgPSBjKDFMLCAyTCwgM0wsIDNMLCA0TCwgNEwsIDRMLCA2TCwgN0wsIDhMLCANCjhMLCA4TCwgOEwsIDEyTCwgMTRMLCAxNUwsIDE3TCwgMTdMLCAxN0wsIDE4TCwgMTlMLCAxOUwsIDIwTCwgDQoyM0wsIDIzTCwgMjNMLCAyNEwsIDI0TCwgMjVMLCAyNkwsIDI3TCwgMjhMLCAyOUwsIDM0TCwgMzZMLCAzNkwsIA0KNDJMLCA0MkwsIDQzTCwgNDNMLCA0NEwsIDQ0TCwgNDRMLCA0NEwsIDQ1TCwgNDZMLCA0OEwsIDQ4TCwgNDlMLCANCjQ5TCwgNTNMLCA1M0wsIDUzTCwgNTRMLCA1NUwsIDU2TCwgNTZMLCA1OEwsIDYwTCwgNjNMLCA2NUwsIDY3TCwgDQo2N0wsIDY4TCwgNzFMLCA3MUwsIDcyTCwgNzJMLCA3MkwsIDczTCwgNzRMLCA3NEwsIDc0TCwgNzVMLCA3NUwsIA0KODBMLCA4MUwsIDgxTCwgODFMLCA4MUwsIDg4TCwgODhMLCA5MEwsIDkzTCwgOTNMLCA5NEwsIDk1TCwgOTVMLCANCjk1TCwgOTZMLCA5NkwsIDk3TCwgOThMLCAxMDBMLCAxMDFMLCAxMDJMLCAxMDNMLCAxMDRMLCAxMDVMLCANCjEwNkwsIDEwN0wsIDEwOEwsIDEwOUwsIDExMEwsIDExMUwsIDExMkwsIDExM0wsIDExNEwsIDExNUwpLCANCiAgICBTdHVkZW50cyA9IGMoNkwsIDhMLCAxMkwsIDlMLCAzTCwgM0wsIDExTCwgNUwsIDdMLCAzTCwgOEwsIA0KICAgIDRMLCA2TCwgOEwsIDNMLCA2TCwgM0wsIDJMLCAyTCwgNkwsIDNMLCA3TCwgN0wsIDJMLCAyTCwgOEwsIA0KICAgIDNMLCA2TCwgNUwsIDdMLCA2TCwgNEwsIDRMLCAzTCwgM0wsIDVMLCAzTCwgM0wsIDNMLCA1TCwgM0wsIA0KICAgIDVMLCA2TCwgM0wsIDNMLCAzTCwgM0wsIDJMLCAzTCwgMUwsIDNMLCAzTCwgNUwsIDRMLCA0TCwgM0wsIA0KICAgIDVMLCA0TCwgM0wsIDVMLCAzTCwgNEwsIDJMLCAzTCwgM0wsIDFMLCAzTCwgMkwsIDVMLCA0TCwgM0wsIA0KICAgIDBMLCAzTCwgM0wsIDRMLCAwTCwgM0wsIDNMLCA0TCwgMEwsIDJMLCAyTCwgMUwsIDFMLCAyTCwgMEwsIA0KICAgIDJMLCAxTCwgMUwsIDBMLCAwTCwgMUwsIDFMLCAyTCwgMkwsIDFMLCAxTCwgMUwsIDFMLCAwTCwgMEwsIA0KICAgIDBMLCAxTCwgMUwsIDBMLCAwTCwgMEwsIDBMLCAwTCkpLCAuTmFtZXMgPSBjKCJEYXlzIiwgIlN0dWRlbnRzIg0KKSwgY2xhc3MgPSAiZGF0YS5mcmFtZSIsIHJvdy5uYW1lcyA9IGMoTkEsIC0xMDlMKSkNCg0KYXR0YWNoKGNhc2VzKQ0KYGBgDQoNCkJlaG9sZCwgdGhlIFBvaXNzb24gZGlzdHJpYnV0aW9uISBOb3RpY2UgdGhlIG1vZGUgYXQgb25lIGV4dHJlbWUgYW5kIHRoZSB0YWlsIG9mZiB0byB0aGUgb3RoZXIgc2lkZS4gVGhpcyBpcyBjaGFyYWN0ZXJpc3RpYyBvZiBQb2lzc29uIGRpc3RyaWJ1dGlvbnMuDQoNCmBgYHtyfQ0KcGxvdChEYXlzLCBTdHVkZW50cywgeGxhYiA9ICJEYXlzIiwgeWxhYiA9ICJTdHVkZW50cyIsIHBjaCA9IDE2KQ0KdGl0bGUobWFpbiA9ICJTcHJlYWQgb2YgVGVlbmFnZSBEaXNlYXNlIFBvc3QtT3V0YnJlYWsiLCBzdWIgPSBOVUxMLA0KICAgICAgbGluZSA9IE5BLCBvdXRlciA9IEZBTFNFKSANCmBgYA0KDQojIyBTZXJpb3VzbHksIEp1c3QgQ2hhbmdlIEZhbWlseQ0KDQpgYGB7cn0NClBvaXNzb25Nb2RlbCA8LSBnbG0oU3R1ZGVudHMgfiBEYXlzLCBwb2lzc29uKQ0Kc3VtbWFyeShQb2lzc29uTW9kZWwpDQpleHAoY29lZihQb2lzc29uTW9kZWwpKQ0KZXhwKGNvbmZpbnQoUG9pc3Nvbk1vZGVsKSkNCmBgYA0KDQpSYXRoZXIgdGhhbiBvZGRzIHJhdGlvcyAod2hpY2ggb25seSBhcHBseSB0byAwLzEgb3V0Y29tZXMpLCB3ZSB1c2UgcmVsYXRpdmUgcmlzayByYXRpb3MgaW4gUG9pc3NvbiByZWdyZXNzaW9uIGZvciBjb3VudCBvdXRjb21lIHZhcmlhYmxlcy4gDQoNCkludGVycHJldGF0aW9uOlRoZSBleHBlY3RlZCBsb2cgY291bnQgZm9yIGVhY2ggdW5pdCBpbmNyZWFzZS9kZWNyZWFzZSAoZGVwZW5kaW5nIG9uIHRoZSBzaWduIG9mIHRoZSBjb2VmZmljaWVudCkgaW4gW291dGNvbWUgdmFyaWFibGVdIGdpdmVuIFtwcmVkaWN0b3IgdmFyaWFibGVdIGlzIFtjb2VmZmljaWVudF0uICANCg0KDQojIFJlZmVyZW5jZXMNCkthYmFjb2ZmLCBSLkkuIEdlbmVyYWxpemVkIExpbmVhciBNb2RlbHMuIFJldHJpZXZlZCBmcm9tIGh0dHA6Ly93d3cuc3RhdG1ldGhvZHMubmV0L2FkdnN0YXRzL2dsbS5odG1sDQpMaWxsaXMsIEQuIEdlbmVyYWxpemVkIExpbmVhciBNb2RlbHMgaW4gUiwgUGFydCA2OiBQb2lzc29uIFJlZ3Jlc3Npb24gZm9yIENvdW50IFZhcmlhYmxlcy4gUmV0cmlldmVkIGZyb20gaHR0cDovL3d3dy50aGVhbmFseXNpc2ZhY3Rvci5jb20vZ2VuZXJhbGl6ZWQtbGluZWFyLW1vZGVscy1pbi1yLXBhcnQtNi1wb2lzc29uLXJlZ3Jlc3Npb24tY291bnQtdmFyaWFibGVzLyAgDQpSb2Ryw61ndWV6LCBHLiA1IEdlbmVyYWxpemVkIExpbmVhciBNb2RlbHMuIFJldHJpZXZlZCBmcm9tIGh0dHA6Ly9kYXRhLnByaW5jZXRvbi5lZHUvUi9nbG1zLmh0bWwNClVDTEE6IFN0YXRpc3RpY2FsIENvbnN1bHRpbmcgR3JvdXAuIFIgRGF0YSBBbmFseXNpcyBFeGFtcGxlczogTG9naXQgUmVncmVzc2lvbi4gUmV0cmlldmVkIGZyb20gaHR0cDovL3d3dy5hdHMudWNsYS5lZHUvc3RhdC9yL2RhZS9sb2dpdC5odG0NCg0KPHNjcmlwdD4NCiAgKGZ1bmN0aW9uKGkscyxvLGcscixhLG0pe2lbJ0dvb2dsZUFuYWx5dGljc09iamVjdCddPXI7aVtyXT1pW3JdfHxmdW5jdGlvbigpew0KICAoaVtyXS5xPWlbcl0ucXx8W10pLnB1c2goYXJndW1lbnRzKX0saVtyXS5sPTEqbmV3IERhdGUoKTthPXMuY3JlYXRlRWxlbWVudChvKSwNCiAgbT1zLmdldEVsZW1lbnRzQnlUYWdOYW1lKG8pWzBdO2EuYXN5bmM9MTthLnNyYz1nO20ucGFyZW50Tm9kZS5pbnNlcnRCZWZvcmUoYSxtKQ0KICB9KSh3aW5kb3csZG9jdW1lbnQsJ3NjcmlwdCcsJ2h0dHBzOi8vd3d3Lmdvb2dsZS1hbmFseXRpY3MuY29tL2FuYWx5dGljcy5qcycsJ2dhJyk7DQoNCiAgZ2EoJ2NyZWF0ZScsICdVQS05ODg3ODc5My0xJywgJ2F1dG8nKTsNCiAgZ2EoJ3NlbmQnLCAncGFnZXZpZXcnKTsNCg0KPC9zY3JpcHQ+