1 Setup

NOTE ON PACKAGES: I use the following packages in this chapter. If you do not have these packages installed, you may use the script below to install them and load their libraries. I use the kable function of the “knitr” package to create a table in section 5.2 B, and I use the dcast function of the “reshape2” package to restructure that table, also in section 5.2 B.

#install.packages("reshape2")
#install.packages("knitr")
library(reshape2)
library(knitr)

Before beginning, you may set your working directory if you desire:

#setwd("C:/Users/Felix/Desktop/R Working Directory")

2 The rnorm Function and Basic Simulation

In this section, I will introduce the rnorm function and provide basic use cases in the context of simulating observations drawn from a normally distributed population.

The rnorm function is used to generate a specified number of “samples” from a normally distributed population with a user-specified mean and standard deviation. As such, the rnorm function is often used to simulate empirical data collected from a population with known mean and standard deviation. The rnorm function uses the following syntax and arguments:

rnorm(n, mean = 0, sd = 1)

Where n is the number of “observations” to be generated, “mean” is the user-specified population mean, and “sd” is the user-specified standard deviation of the population. To begin to show the usefulness of this function, I will introduce a few basic examples.

2.1 A. Using rnorm(): Basic Examples and the set.seed() Function.

Let’s use the rnorm function to simulate 10 observations from a population with a mean of 15 and a standard deviation of 2. I will store these 10 generated values in an object I call “rnorm.example” in order to be able to find the mean of these values (and potentially perform other manipulations and calculations on them later on).

rnorm.example<-rnorm(10, mean=15, sd=2)
rnorm.example
##  [1] 12.63554 14.68902 17.56492 12.15005 17.01833 15.01955 13.22323
##  [8] 15.45385 16.45164 12.63038
mean(rnorm.example)
## [1] 14.68365

It is important to note that while these observations are drawn from a normally distributed population with a mean of 15 and a standard deviation of 2, the mean of this sample is NOT 15, as seen above. The rnorm function does not create a normally distributed sample with specified mean, sd, and number of observations - it draws observations from a theoretical population with specified parameters.

Note that performing the same rnorm operation will result in a different set of values being generated each time:

rnorm(10, mean = 15, sd=2)
##  [1] 12.89967 18.11219 12.27217 12.36293 14.21295 16.19919 14.48325
##  [8] 12.78139 14.30000 14.81097
rnorm(10, mean = 15, sd=2)
##  [1] 15.122720 15.346049 13.362262  9.892635 14.313268 17.725910 14.653110
##  [8] 17.224562 14.288682 13.297050

By contrast, running the named object “rnorm.example” repeatedly returns the same set of values each time:

rnorm.example
##  [1] 12.63554 14.68902 17.56492 12.15005 17.01833 15.01955 13.22323
##  [8] 15.45385 16.45164 12.63038
rnorm.example
##  [1] 12.63554 14.68902 17.56492 12.15005 17.01833 15.01955 13.22323
##  [8] 15.45385 16.45164 12.63038

However, re-running the original expression and then calling the object will result in the values of “rnorm.example” being changed each time the expression is re-run:

rnorm.example<-rnorm(10, mean=15, sd=2)
rnorm.example
##  [1] 15.127496 17.443617 16.709997 14.789930 13.768781  9.610574 12.011626
##  [8] 15.164009 17.854578 13.936753
rnorm.example<-rnorm(10, mean=15, sd=2)
rnorm.example
##  [1] 17.85742 16.76920 11.97688 19.16544 14.06496 16.72299 18.01710
##  [8] 13.43506 13.06632 12.10678

As such, if you want to obtain same set of numbers each time you run a chunk of code, you must use the set.seed() function.

If you want to generate the same numbers each time you run an rnorm expression, you can use the set.seed function, which uses the following syntax: set.seed(seed)

In which “seed” is an integer that sets the seed, such that specifying that same integer later will recall the particular set of values associated with that seed integer.

To see how setting a given seed will produce the same values every time that seed is specified, even if the expression is re-run, let’s reproduce the previous code chunk but specify a seed of “1” for “rnorm.example”:

set.seed(1)
rnorm.example<-rnorm(10, mean=15, sd=2)
rnorm.example
##  [1] 13.74709 15.36729 13.32874 18.19056 15.65902 13.35906 15.97486
##  [8] 16.47665 16.15156 14.38922
rnorm.example<-rnorm(10, mean=15, sd=2)
set.seed(1)
rnorm.example<-rnorm(10, mean=15, sd=2)
rnorm.example
##  [1] 13.74709 15.36729 13.32874 18.19056 15.65902 13.35906 15.97486
##  [8] 16.47665 16.15156 14.38922

Unlike in the previous chunk of code, this time re-running the expression does not change the numbers in “rnorm.example”, since we set a seed. What happens when we specify a seed other than 1?

set.seed(2)
rnorm.example<-rnorm(10, mean=15, sd=2)
rnorm.example
##  [1] 13.20617 15.36970 18.17569 12.73925 14.83950 15.26484 16.41591
##  [8] 14.52060 18.96895 14.72243
set.seed(2)
rnorm.example<-rnorm(10, mean=15, sd=2)
rnorm.example
##  [1] 13.20617 15.36970 18.17569 12.73925 14.83950 15.26484 16.41591
##  [8] 14.52060 18.96895 14.72243

Setting a seed of 2 will create a different set of 10 generated values, which can be reproduced each time the seed 2 is specified. Thus, different seed numbers can be used to store different sets of values for the same expression. The seed numbers I use throughout this chapter were chosen by me at random and hold no symbolic meaning whatsoever. In the case of this chapter, the set.seed() function is useful as it ensures that that, whenever I specify a seed, the same numbers will appear each time the full chapter code is run.

2.2 B. Using rnorm for Basic Simulation

Let’s use the rnorm function to simulate some data in a real-world context!

To set the stage for the scenario I want to simulate, I will first need to introduce a new construct:

“[REDACTED], it won’t work!”.

This construct describes the mixture of frustration, despair, and resignation that students in R courses experience when they encounter a perplexing problem in R, whether it be packages that refuse to install, functions that yield inscrutable error messages, or code that runs perfectly on a neighboring computer but stubbornly refuses to run on yours. We can use the rnorm function to simulate the number of instances of this experience over the course of a semester-long R class.

More specifically, we will use rnorm to simulate the data we might collect in surveying the number of instances of “[REDACTED], it won’t work!” that Mac and PC users might encounter over the course of the first and second half of a semester-long R course. Generally, when simulating data, we either [WHAT GOES HERE??? A WORD OR PHRASE IS MISSING] on known population parameters or on theoretically driven assumptions regarding the population parameters. The data I will generate are based on the following assumptions (derived from informal observation of my R class) regarding what observations would look like:

  1. Instances of “[REDACTED], it won’t work!” encountered during R use may be caused either by user error/unfamiliarity with the R program or by R program bugs/compatibility issues.

  2. As students become more familiar with the R program, user error decreases, leading to fewer instances of “[REDACTED], it won’t work!” stemming from user error for both Mac and PC users in 2nd half of semester compared to 1st half.

  3. R studio is better-optimized for PC than for Mac, leading to more instances of “[REDACTED], it won’t work!” caused by program errors and compatibility issues for Mac users than for PC users.

Based on these assumptions, given instances of “[REDACTED], it won’t work!” as the dependent variable, I would expect to observe a main effect of Time (more instances in 1st half than 2nd half of semester) and of User Type (more instances for Mac users than PC users). Let’s use the rnorm function to simulate this pattern of results. Note that other factors that might influence the dependent variable, such as individual R proficiency and psychological effects of R course professors frequently disparaging Macs, are not modeled here.

set.seed(103)
MacUsers1stHalf <- rnorm(100, mean = 500, sd = 35)
PCUsers1stHalf <- rnorm(100, mean = 450, sd = 35)
MacUsers2ndHalf <-rnorm(100, mean = 150, sd = 35)
PCUsers2ndHalf <- rnorm (100, mean = 100, sd=35)

As you see in the script above, I set the averages to simulate the hypothesized pattern of results. As I had no predictions regarding the distribution of data, I have no strong theoretical rationale for setting a standard deviation of 35. However, keeping the default standard deviation value of 1 would have yielded unnatural-looking simulated data, as the distribution of data would have been too tightly clustered around the mean. I used the set.seed function to ensure that the same values will be generated each time this code is run. Again, why not just use the object like you did with rnorm.example? Why always use set.seed? If you give the explanation before this mention of it, I wouldn’t be questioning it down here. The next section discusses the rep function. Once we have learned the basics of the rep function, we will be able to do more with the observations we have simulated.

3 The rep Function and its Applications

In this section, I will introduce the rep function and provide a basic use case in the context of creating a data frame from the simulated data we have generated. The rep function replicates the values in a given object (a vector or a function) a specified number of times. The rep function uses the following syntax and arguments:

        ```rep(x, ...)```
        

Where “x” is the value to be replicated or the vector containing values to be replicated, and “…” denotes additional arguments, such as “times=” (which specifies the number of times to replicate “x”), “each=” (which specifies the number of times to repeat each element of “x”), and “len=” (which specifies the desired length of an output vector). Below, I go over some very basic examples of each of these uses of the rep function.

3.1 A. Examples of simple replication of individual values and vectors

3.1.0.1 The simplest use of the replicate function is to replicate a text or numeric value a set number of times. To begin, let’s replicate the number 5 ten times:

rep(5, 10)
##  [1] 5 5 5 5 5 5 5 5 5 5

As you see above, entering a value n such that the sytax reads “rep(x,n)” defaults to the same meaning as the syntax “rep(x, times=n)”. Aside from individual values, we can also replicate a text label in the same manner:

rep("Kitty Cat", 10)
##  [1] "Kitty Cat" "Kitty Cat" "Kitty Cat" "Kitty Cat" "Kitty Cat"
##  [6] "Kitty Cat" "Kitty Cat" "Kitty Cat" "Kitty Cat" "Kitty Cat"

One possible use for replicating individual values or text labels is populating columns for continuous or categorical data in a data frame. I explore this use in section 5.2 B below.

Aside from replicating individual text or numeric values, the rep function is commonly used to replicate (elements within) vectors. Below, I create a simple vector of seven numeric values and then replicate that vector, as a whole, 3 times.

ExampleRepVector<-c(1,2,5,7,7,8,10)
rep(ExampleRepVector, 3)
##  [1]  1  2  5  7  7  8 10  1  2  5  7  7  8 10  1  2  5  7  7  8 10

We can also replicate each element of this vector a set number of times. Below, I first replicate each element of the vector twice. Next, I replicate each element twice, while also replicating the vector as a whole (with each element present twice) three times.

rep(ExampleRepVector, each=2)
##  [1]  1  1  2  2  5  5  7  7  7  7  8  8 10 10
rep(ExampleRepVector, times=3, each=2)
##  [1]  1  1  2  2  5  5  7  7  7  7  8  8 10 10  1  1  2  2  5  5  7  7  7
## [24]  7  8  8 10 10  1  1  2  2  5  5  7  7  7  7  8  8 10 10

Finally, let’s see what happens when we specify an output vector of a different length than the number of terms in the original vector (7):

20 terms:

rep(ExampleRepVector, len=20)
##  [1]  1  2  5  7  7  8 10  1  2  5  7  7  8 10  1  2  5  7  7  8

5 terms:

rep(ExampleRepVector, len=5)
## [1] 1 2 5 7 7

When outputting to a vector longer than the original vector, the original vector is replicated until the output vector has been filled. In our case, filling the 20 term output vector requires the original vector to be replicated twice, with an incomplete 3rd replication of only the first six terms.

When outputting to a vector shorter than the original vector, the output vector is filled before all of the values of the original vector can be replicated. In our case, the original vector has 7 values and the output vector only has room for 5, so the last 2 values are not replicated.

3.2 B. Using the rep Function to Create Data Frames

3.2.0.1 Now that we’ve gone over the basic uses of the rep function, let’s use the rep function to help us create a data frame for the simulated data we created in section 5.1 . Recall that we have one between-subjects variable, UserType (Mac user or PC user), and one within-subjects variable, Time (1st and 2nd Half of Semester). As we have 100 Mac Users and 100 PC users, our total number of subjects is 200.

In creating the data frame, I will use the rep function to replicate the labels of each level of each IV across the relevant range of subjects. Below, I set these arguments and create the data frame:

n.MacUsers=100
n.PCUsers=100
N=n.MacUsers + n.PCUsers

SimulatedData <- data.frame(
  Subject = c(seq(1,N),seq(1,N)),
  Time = c(rep("1st Half", N), rep("2nd Half", N)),
  UserType = c(rep(c(rep("Mac User", n.MacUsers), rep("PC User", n.PCUsers)),2)),
  Instances = c(MacUsers1stHalf,PCUsers1stHalf, 
                MacUsers2ndHalf,PCUsers2ndHalf
           
  )
)

Using the rep function to denote which level of the within- and between-subjects IV each data point is associated with, as in the above data frame syntax, allows us to see which values correspond to Mac Users and PC Users, and to the 1st Half and 2nd Half of the semester. Let’s make a table of these data. As there are a total of 400 columns in the full table, I will be using the head() function to display only the first several columns. I will be using the kable function in the knitr package to produce this table; further documentation and help for this package can be found at the following URL: http://yihui.name/knitr/

knitr::kable(head(SimulatedData))
Subject Time UserType Instances
1 1st Half Mac User 472.4909
2 1st Half Mac User 501.9159
3 1st Half Mac User 458.9604
4 1st Half Mac User 494.1441
5 1st Half Mac User 434.7239
6 1st Half Mac User 495.7846

This table could be more easily interpreted if the two levels of Time, 1st Half and 2nd Half, were represented on separate columns. We can use the dcast() function (part of the “reshape2” package) to change the structure of the data frame and then create a new table. Below, I use the dcast function to restructure the data frame by splitting the variable time into two columns corresponding to the 2 levels of the variable: First Half and Second Half. Again, I use the head() function to display only the first several columns of a table that has 200 columsn in total. Further documentation and help with the reshape 2 package can be found at the following URL: http://seananderson.ca/2013/10/19/reshape.html

SimulatedData.Wide<-dcast(SimulatedData, Subject + UserType ~ Time, value.var="Instances")
knitr::kable(head(SimulatedData.Wide))
Subject UserType 1st Half 2nd Half
1 Mac User 472.4909 142.7769
2 Mac User 501.9159 154.4736
3 Mac User 458.9604 111.5123
4 Mac User 494.1441 189.0489
5 Mac User 434.7239 111.8786
6 Mac User 495.7846 138.1905

As you can see, the variable “Time” has now been split into two columns, “1st Half” and “2nd Half”. To see the full table, either with or without the split, view the object in the Global Environment.

4 The sample Function

In this section, I will introduce the sample function, provide examples of basic use, and discuss sampling with and without replacement and bootstrapping. The sample function is most commonly used to take a sample of the elements of a vector, which can be done either with or without replacement. The sample function uses the following syntax and arguments:

sample(x, size, replace = FALSE)

Where “x” is a vector of elements to choose from, “size” is the number of elements to be sampled, and “replace=” can be set to either “TRUE” or “FALSE” depending on whether you wish to sample with or without replacement. Specifying replace = “TRUE” will samle with replacement, whereas specifying replace = “FALSE” will sample without replacement. Below, I go over basic uses of the “sample” function to sample with and without replacement.

4.1 A. Basic Examples Using the sample Function

First, let’s create a vector of values which does not include any repeated elements. This will make it easier to see the difference between sampling with and without replacement.

ExampleSampleVector<-c(1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20)

Let’s sample fifteen values from this vector WITHOUT replacement

sample(ExampleSampleVector,15,replace = FALSE)
##  [1] 18  4  6  8  3 13 14  5 10 12 15 16  9 20  7

As you can see, just as no value appeared in the original vector more than once, no value appears in our sample more than once. Sampling without replacement means that once a value has been sampled, it is “removed” from the original vector and cannot be randomly sampled again.

Now, let’s sample fifteen values from this vector WITH replacement

sample(ExampleSampleVector,15,replace = TRUE)
##  [1] 11 11 15  6 17  9  1 14 10 16  8 13 15 18 13

This time, it is likely that there is at least one duplicate value in our sample. Sampling with replacement means that sampled values are “replaced” in the original vector and thus may be randomly sampled again.

When drawing random samples of a few observations, a different set of observations will generally be sampled each time you run the sample function. If you want to keep the same set of sampled observations in order to perform further manipulations using that particular set, you can use the set.seed function here as well:

set.seed(2); sample(ExampleSampleVector, 15)
##  [1]  4 14 11  3 16 15  2 18  6  7 12 17 13 19  9
set.seed(2); sample(ExampleSampleVector, 15)
##  [1]  4 14 11  3 16 15  2 18  6  7 12 17 13 19  9

As you can see, the two sampled sets are identical, thanks to the set.seed function. I can use this seed to recall this particular set of fifteen sampled values if I want to use them in the future.

4.2 B. Using the sample Function and Bootstrapping on our Simulated Mac and PC User Data

Now, we will discuss sampling in the context of our simulated data, and introduce the concept of bootstrapping. Bootstrapping involves resampling from a population or set of values a specified number of times in order for the mean of your samples to more closely approximate the mean of the population or original set of values. While there are several ways to perform bootstrapped sampling in R, the simplest is to use the replicate function, which I will discuss only as it pertains to bootrapping. The replicate function repeats a given function a specified number of times, using the following syntax and arguments:

replicate(n, expr)

Where n is the number of desired replications and “expr” is the expression to be replicated, usually an object already entered in the global environment.

Boostrapping is often used when sampling from a normally distributed population. Here, I discuss sampling and bootstrapping samples from a larger set of observations. In the next section, 5.4, I discuss sampling from a population in the context of demonstrating the Law of Large Numbers.

Let’s sample ten values from our data corresponding to Mac Users in the first half of the semester, and compare the mean of that sample to the mean of the dataset being sampled from:

mean(MacUsers1stHalf)
## [1] 500.3287
set.seed(14)
MacUsersSample.1<- sample(MacUsers1stHalf, 10)
MacUsersSample.1
##  [1] 525.9583 451.4742 453.5782 509.6634 482.8383 465.0211 451.4292
##  [8] 509.0998 507.0654 499.6237
mean(MacUsersSample.1)
## [1] 485.5752

A random sample of ten values is not always representative of the larger set from which it is drawn. In this case, we have randomly sampled ten values from the 100 values that represent the number of instances of “[REDACTED], it won’t work!” of 100 Mac Users over the first half of semester. We know that the average number of instances for this group is 500.33, but the average of our ten sampled values is roughly 486. We can improve the accuracy of the mean estimate obtained through sampling by using bootstrapping.

set.seed(993)
TenSampleBootstrapNR<-replicate(10, sample(MacUsers1stHalf, 10, replace = FALSE))
TenSampleBootstrapNR
##           [,1]     [,2]     [,3]     [,4]     [,5]     [,6]     [,7]
##  [1,] 482.8383 474.9895 458.9604 504.5545 553.3036 466.6709 532.2691
##  [2,] 501.3101 494.6208 467.3723 446.8776 446.8776 467.3774 553.3036
##  [3,] 510.7601 537.5814 482.2247 483.4794 528.7878 511.9421 479.1027
##  [4,] 504.0899 527.7317 479.1027 472.4909 501.3101 516.9374 494.1441
##  [5,] 494.1441 501.3101 494.1441 464.3050 498.2764 461.7992 484.4387
##  [6,] 483.4794 561.6610 516.2847 509.0998 522.1641 527.7317 457.2586
##  [7,] 522.1641 505.9178 490.5675 560.1585 486.8711 535.5993 458.9604
##  [8,] 434.7239 536.5534 522.9261 504.0382 458.3491 494.1441 494.6208
##  [9,] 466.6709 501.1626 458.3491 495.7846 511.9421 536.5534 561.6610
## [10,] 461.7992 491.9022 448.3091 529.8700 497.9852 465.0211 453.4769
##           [,8]     [,9]    [,10]
##  [1,] 467.3774 453.4769 464.0428
##  [2,] 522.9261 479.1027 541.6556
##  [3,] 569.8026 463.1552 482.7156
##  [4,] 502.8677 446.8776 510.7601
##  [5,] 458.9604 495.0046 509.0998
##  [6,] 546.8735 505.9178 569.8026
##  [7,] 560.8189 448.3091 515.2273
##  [8,] 501.3091 465.0211 516.2847
##  [9,] 466.7099 524.3736 465.0211
## [10,] 497.9852 451.4742 424.9227
MeanTenSampleBootstrapNR<-replicate(10, mean(sample(MacUsers1stHalf, 10, replace = FALSE)))
MeanTenSampleBootstrapNR
##  [1] 486.6228 507.5105 496.0812 505.0298 491.6476 506.2274 489.8495
##  [8] 496.4604 489.9878 491.1184
mean(MeanTenSampleBootstrapNR)
## [1] 496.0535

Taking 10 bootrapped samples of ten values, without replacement each yields an average of roughly 496 - an improvement. Let’s try the same thing for sampling with replacement:

set.seed(82)
TenSampleBoostrapWR<-replicate(10, sample(MacUsers1stHalf,10,replace = TRUE))
TenSampleBoostrapWR
##           [,1]     [,2]     [,3]     [,4]     [,5]     [,6]     [,7]
##  [1,] 501.1626 515.2273 524.3736 504.0899 424.9227 499.6237 560.4686
##  [2,] 461.7992 501.9159 525.9583 537.5814 527.7317 516.0727 467.3774
##  [3,] 535.5993 465.0211 511.9421 477.8833 453.5782 501.1626 461.7992
##  [4,] 507.0654 479.1027 516.9374 479.1027 451.4742 537.5814 458.3491
##  [5,] 544.4890 504.0382 465.0211 463.1552 465.0211 509.6634 494.6208
##  [6,] 532.2691 490.5675 461.7992 532.2691 491.4081 560.1585 479.1027
##  [7,] 461.7992 525.9583 544.4890 505.9178 541.6556 510.7601 504.7141
##  [8,] 509.1655 504.7141 516.9374 511.0911 451.4549 482.7156 509.1655
##  [9,] 560.8189 472.8796 491.9022 497.9852 467.3774 451.4549 463.1552
## [10,] 491.4081 472.4909 468.9095 482.7156 434.7239 484.1997 522.1641
##           [,8]     [,9]    [,10]
##  [1,] 528.7878 511.0911 446.8776
##  [2,] 524.3736 486.8711 453.4769
##  [3,] 527.7317 446.8776 501.1626
##  [4,] 484.4387 477.8833 466.6709
##  [5,] 509.1655 525.9583 466.6709
##  [6,] 491.4081 528.7878 510.7601
##  [7,] 474.9895 467.3723 498.8165
##  [8,] 463.1552 453.5452 497.9852
##  [9,] 434.7239 536.5534 504.2742
## [10,] 483.4794 453.5782 528.7878
MeanTenSampleBootstrapWR<-replicate(10, mean(sample(MacUsers1stHalf,10,replace = TRUE)))
MeanTenSampleBootstrapWR
##  [1] 514.7425 498.8315 491.6189 494.2262 497.2734 497.1704 498.6680
##  [8] 502.5443 503.0764 492.2138
mean(MeanTenSampleBootstrapWR)
## [1] 499.0365

Here, our mean is roughly 499. Again, this is an improvement over taking only one sample. However, we can guarantee ever more accurate estimates of the mean by taking even more bootstrapped samples, as seen in the following example.

Now, let’s try estimating the mean based on 100 bootstrapped samples of ten observations each, first without and then with replacement:

set.seed(90)
HundredSampleBootstrapNR<-replicate(100, sample(MacUsers1stHalf, 10, replace = FALSE))
MeanHundredSampleBootsrapNR<-replicate(100, mean(sample(MacUsers1stHalf, 10, replace = FALSE)))
MeanHundredSampleBootsrapNR
##   [1] 483.9443 482.1300 501.8791 497.3724 521.8733 495.3230 504.2639
##   [8] 502.6423 490.4462 499.9054 505.3837 496.8223 498.4532 504.4094
##  [15] 493.6311 484.4697 506.3115 507.2337 502.4622 506.0444 493.9876
##  [22] 479.9301 505.4607 485.3938 499.5878 498.2262 511.7374 501.2629
##  [29] 500.0467 487.8291 504.6782 514.9830 499.7610 485.8169 516.1472
##  [36] 491.6890 499.5895 505.9952 484.8721 505.3810 516.2393 501.0103
##  [43] 494.4457 503.0997 497.2621 515.9588 508.0401 501.7428 516.8377
##  [50] 500.4021 500.2011 500.6698 507.9374 516.0995 511.6492 497.5592
##  [57] 512.4951 499.4033 483.3637 496.6662 511.4488 499.6909 503.0627
##  [64] 511.9006 485.8086 489.7278 499.6084 493.6650 497.9322 494.2967
##  [71] 492.8505 500.1848 501.2082 513.1225 497.2403 484.4841 514.7898
##  [78] 495.1924 501.3294 498.0645 490.8601 496.2211 487.8071 497.7390
##  [85] 502.5861 498.2383 472.8172 508.5017 498.8662 500.0314 494.8469
##  [92] 482.0513 505.2282 500.5006 501.6754 513.7240 513.7879 491.4527
##  [99] 496.3572 497.7718
mean(MeanHundredSampleBootsrapNR)
## [1] 499.7513
set.seed(92)
HundredSampleBootstrapWR<-replicate(100, sample(MacUsers1stHalf, 10, replace = TRUE))
MeanHundredSampleBootstrapWR<-replicate(100, mean(sample(MacUsers1stHalf, 10, replace = TRUE)))
MeanHundredSampleBootstrapWR
##   [1] 501.6860 497.1856 499.6808 495.2185 524.2279 494.7761 507.2499
##   [8] 490.6339 516.4108 478.7056 501.4649 500.1753 496.9795 498.5235
##  [15] 489.4741 506.9105 501.8296 483.6765 500.2106 492.0935 502.8805
##  [22] 501.4384 491.0846 487.5533 494.4862 491.7782 495.9216 499.3074
##  [29] 493.5227 494.7585 490.3979 502.3776 498.2886 483.9435 502.6134
##  [36] 499.6300 507.9872 501.9230 493.7481 500.8062 492.6423 490.5927
##  [43] 510.2734 481.4542 498.5302 515.0135 505.5613 493.5074 515.0532
##  [50] 500.5499 503.1455 498.9470 492.1895 499.8119 482.2462 514.1567
##  [57] 498.2753 508.2647 488.7334 508.0879 490.0728 493.2714 490.4271
##  [64] 498.7075 496.0585 507.2410 502.0665 497.6320 503.8993 509.9675
##  [71] 479.2922 512.2531 504.1145 494.1574 506.8482 507.0131 489.9587
##  [78] 503.6567 478.6168 504.7695 482.8386 500.7073 498.8247 512.1134
##  [85] 499.9993 483.4650 510.6256 489.4198 508.0756 508.6300 504.6548
##  [92] 513.8521 497.0976 497.8362 501.1025 504.4786 504.7135 520.8636
##  [99] 513.7015 492.8148
mean(MeanHundredSampleBootstrapWR)
## [1] 499.2443

Taking 100 bootstrapped samples further improves our estimate of the mean of the group we are sampling from. Now, our mean is very close to 500, the true value. The larger the size of the population being sampled from, the less of a difference there is between sampling with and without replacement. In our case, as there were 100 values in the group being sampled from and we sampled ten values at a time, it didn’t make a large difference whether we sampled with or without replacement.

Seeing as the rnorm function generates a sample of a specified number of values from a hypothetical normal distribution with a user-specified mean and standard deviation, bootstrapping simulations are often performed using the rnorm function as well.

5 Applications to General Statistical Problems

5.1 A. Demonstrating the Law of Large Numbers

Acknowledgment: This section drew inspiration from a demonstration of the Law of Large numbers created by Pedro F. Quintana-Ascencio and James Angelo at UCF, which can be found at the following URL: http://pascencio.cos.ucf.edu/classes/Methods/R%20demos/R%20Demonstration%20Law%20Large%20numbers%202012.pdf

The Law of Large Numbers states that as the number of observations in a given random sample of the population increases, the mean of that sample approaches the true mean value of the population.

A simple corollary can be made to the act of flipping a coin. We know that the true rate of each outcome, heads or tails, is 0.5. However, if we flip a coin five times, we could easily obtain a result of four heads and 1 tails, yielding a rate of 0.8 for heads and 0.2 for tails. In this case, the mean estimated from a sample of five would be highly innaccurate. However, if we flip a coin one thousand times, it is virtually impossible to obtain such an inaccurate estimate of the true rate of each outcome. Flipping a coin one thousand times would yield a rate very close to 0.5 for both heads and tails.

Let’s demonstrate the Law of Large Numbers in R. To begin, we need to establish a population to sample from. I chose to use the distribution of Japanese mens’ heights as my population, as Japan was the most prominent country for which I could find both a mean and a standard deviation for height (see http://www.mext.go.jp/b_menu/toukei/001/022/2004/002.pdf). Below, I enter the mean of the population, 172 centimeters, as an object “mu”, and the standard deviation, 5.42 centimeters, as an object “sigma”. Mu is the Greek letter used to denote population mean and sigma is the Greek letter used to denote population standard deviation

mu = 172
sigma = 5.42

To demonstrate the Law of Large numbers, I will sample from this population 50 times, with the number of observations per sample increasing from 1 to 50 in increments of one. Once all of the samples have been collected, I will plot the average of each sample to observe whether the distribution of means changes as the number of samples increases.

First, I will create an empty vector of length 50, which will eventually be populated by the means of my 50 samples:

Vector_of_Means <- rep(0,50)

Checking the vector, we can see that it is currently populated by 50 zeroes:

Vector_of_Means
##  [1] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## [36] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Next, I need to start populating the vector. I will construct the vector such that the 1st value will be the mean of a sample of 1 observation, the 2nd value will be the mean of a sample of 2 obsrvations, etc. - all the way to the 50th value which is the mean of a sample of 50 observations. To begin, I need to take a sample of one, find the mean, and enter the mean into the vector of averages. I combine these steps into one function below:

Vector_of_Means[1]<-mean(rnorm(1,mean=mu,sd=sigma))

Now, I have to repeat the process for samples of 2 to 50 values to populate the 2nd through the 50th slot in the vector. To do this, I could write 49 more expressions like the one above, but a more elegant solution is to use a “for loop to accomplish the same thing. For help with understanding different types of loops in R, see Chapter 3: Loops and Logicals. In the loop below, I am specifying that for a value of i from 2 to 50 (the length of the vector), the ith value of the vector should be the mean of a sample of i values, with the mean and sd establised previously:

for (i in 2:length(Vector_of_Means)) {
    Vector_of_Means[i] <- mean(rnorm(i, mean=mu, sd=sigma))
}

Checking the vector, we can see that it is now populated by the 50 values we have generated:

Vector_of_Means
##  [1] 172.4067 174.4181 166.8237 173.8780 174.8917 169.7871 175.6439
##  [8] 171.2992 174.9572 171.1308 172.4583 172.1380 170.9901 174.8588
## [15] 171.2705 173.6047 171.6975 172.3292 171.5491 172.9344 172.4017
## [22] 171.0498 170.7853 172.5667 172.7151 173.8499 171.3387 171.9326
## [29] 172.0962 171.6051 171.9281 172.2289 173.1625 172.1153 172.4185
## [36] 172.1944 173.4946 173.1161 172.6899 171.9070 173.1682 170.0475
## [43] 172.6212 173.1910 171.4739 172.0959 172.3849 172.6251 172.7516
## [50] 171.6788

Finally, I create a scatterplot of the means, with Sample Size on the x axis and Centimeters on the Y axis. The blue line indicates the true mean of the population, 172 centimeters. For help with graphing and scatterplots, see Chapter 10: GGplot 1 (Scatterplots and Density).

plot(seq(1,50), Vector_of_Means, xlab = "Sample Size", ylab = "Centimeters")
abline(h=mu, col="blue")

From the scatterplot, it is clear that the distribution of points (sample means) clusters closer and closer to the true population mean as sample size increases, demonstrating the Law of Large Numbers. Increasing sample size further would continue this trend.

5.2 B. Simulating a Poisson Distribution

A Poisson distribution is a discrete probability distribution that expresses the probability of a given number of events occurring in a given interval of time, with the requirement that these events occur with a known average rate and that the probability of each event is independent of the probability of the previous events. In a Poisson distribution the variable lambda, sometimes called the rate parameter, represents the average number of events in a given time interval.

The rpois() function can be used to simulate a Poisson distribution in R using the following syntax:

```rpois(n, lambda)```

Where n is the number of random Poisson numbers to generate and lambda is the aforementioned rate parameter of the distribution.

Let’s simulate a Poisson distribution for a real world scenario. We can use a Poisson distribution to simulate the number of emails that a typical UIC psychology graduate student receives each day (including UIC MASSMAIL) over a 100 day period. Because the probability of each email received is usually independent of previous emails received, a Poisson distribution is a good choice to model this data. Let’s simulate data for a typical student for 100 days, assuming an average rate of 30 emails per day (lambda=30):

PoissonEmails<-rpois(100, 30)

To visualize these data, let’s make a scatterplot:

plot(PoissonEmails,xlab="Day Number", ylab = "Number of Emails Received")
abline(h=30, col="blue")

The plot above shows the simulated data representing the number of emails received each day over the 100 day period. The blue line represents the average rate of 30 which I set for this distribution. As expected, data points for specific days fall both above and below this specified average daily rate.

LS0tDQp0aXRsZTogIkNoYXB0ZXIgNzogU2FtcGxpbmcgYW5kIFJlcGxpY2F0aW9uIg0KYXV0aG9yOiAiRmVsaXggUGFtYnVjY2lhbiINCm91dHB1dDoNCiAgaHRtbF9kb2N1bWVudDoNCiAgICB0aGVtZTogY2VydWxlYW4NCiAgICBoaWdobGlnaHQ6IHRleHRtYXRlDQogICAgZm9udHNpemU6IDhwdA0KICAgIHRvYzogdHJ1ZQ0KICAgIG51bWJlcl9zZWN0aW9uczogdHJ1ZQ0KICAgIGNvZGVfZG93bmxvYWQ6IHRydWUNCiAgICB0b2NfZmxvYXQ6DQogICAgICBjb2xsYXBzZWQ6IGZhbHNlDQotLS0NCg0KIyBTZXR1cA0KTk9URSBPTiBQQUNLQUdFUzogSSB1c2UgdGhlIGZvbGxvd2luZyBwYWNrYWdlcyBpbiB0aGlzIGNoYXB0ZXIuICBJZiB5b3UgZG8gbm90IGhhdmUgdGhlc2UgcGFja2FnZXMgaW5zdGFsbGVkLCB5b3UgbWF5IHVzZSB0aGUgc2NyaXB0IGJlbG93IHRvIGluc3RhbGwgdGhlbSBhbmQgbG9hZCB0aGVpciBsaWJyYXJpZXMuICBJIHVzZSB0aGUga2FibGUgZnVuY3Rpb24gb2YgdGhlICJrbml0ciIgcGFja2FnZSB0byBjcmVhdGUgYSB0YWJsZSBpbiBzZWN0aW9uIDUuMiBCLCBhbmQgSSB1c2UgdGhlIGRjYXN0IGZ1bmN0aW9uIG9mIHRoZSAicmVzaGFwZTIiIHBhY2thZ2UgdG8gcmVzdHJ1Y3R1cmUgdGhhdCB0YWJsZSwgYWxzbyBpbiBzZWN0aW9uIDUuMiBCLg0KDQpgYGB7cn0NCiNpbnN0YWxsLnBhY2thZ2VzKCJyZXNoYXBlMiIpDQojaW5zdGFsbC5wYWNrYWdlcygia25pdHIiKQ0KYGBgDQoNCmBgYHtyfQ0KbGlicmFyeShyZXNoYXBlMikNCmxpYnJhcnkoa25pdHIpDQpgYGANCg0KQmVmb3JlIGJlZ2lubmluZywgeW91IG1heSBzZXQgeW91ciB3b3JraW5nIGRpcmVjdG9yeSBpZiB5b3UgZGVzaXJlOg0KYGBge3J9DQojc2V0d2QoIkM6L1VzZXJzL0ZlbGl4L0Rlc2t0b3AvUiBXb3JraW5nIERpcmVjdG9yeSIpDQpgYGANCg0KDQojIFRoZSBybm9ybSBGdW5jdGlvbiBhbmQgQmFzaWMgU2ltdWxhdGlvbg0KSW4gdGhpcyBzZWN0aW9uLCBJIHdpbGwgaW50cm9kdWNlIHRoZSBybm9ybSBmdW5jdGlvbiBhbmQgcHJvdmlkZSBiYXNpYyB1c2UgY2FzZXMgaW4gdGhlIGNvbnRleHQgb2Ygc2ltdWxhdGluZyBvYnNlcnZhdGlvbnMgZHJhd24gZnJvbSBhIG5vcm1hbGx5IGRpc3RyaWJ1dGVkIHBvcHVsYXRpb24uICANCg0KVGhlIHJub3JtIGZ1bmN0aW9uIGlzIHVzZWQgdG8gZ2VuZXJhdGUgYSBzcGVjaWZpZWQgbnVtYmVyIG9mICJzYW1wbGVzIiBmcm9tIGEgbm9ybWFsbHkgZGlzdHJpYnV0ZWQgcG9wdWxhdGlvbiB3aXRoIGEgdXNlci1zcGVjaWZpZWQgbWVhbiBhbmQgc3RhbmRhcmQgZGV2aWF0aW9uLiAgQXMgc3VjaCwgdGhlIHJub3JtIGZ1bmN0aW9uIGlzIG9mdGVuIHVzZWQgdG8gc2ltdWxhdGUgZW1waXJpY2FsIGRhdGEgY29sbGVjdGVkIGZyb20gYSBwb3B1bGF0aW9uIHdpdGgga25vd24gbWVhbiBhbmQgc3RhbmRhcmQgZGV2aWF0aW9uLiBUaGUgcm5vcm0gZnVuY3Rpb24gdXNlcyB0aGUgZm9sbG93aW5nIHN5bnRheCBhbmQgYXJndW1lbnRzOg0KICAgICAgICAgICAgDQpgYGBybm9ybShuLCBtZWFuID0gMCwgc2QgPSAxKWBgYA0KICAgICAgICAgICAgDQpXaGVyZSBuIGlzIHRoZSBudW1iZXIgb2YgIm9ic2VydmF0aW9ucyIgdG8gYmUgZ2VuZXJhdGVkLCAibWVhbiIgaXMgdGhlIHVzZXItc3BlY2lmaWVkIHBvcHVsYXRpb24gbWVhbiwgYW5kICJzZCIgaXMgdGhlIHVzZXItc3BlY2lmaWVkIHN0YW5kYXJkIGRldmlhdGlvbiBvZiB0aGUgcG9wdWxhdGlvbi4gIFRvIGJlZ2luIHRvIHNob3cgdGhlIHVzZWZ1bG5lc3Mgb2YgdGhpcyBmdW5jdGlvbiwgSSB3aWxsIGludHJvZHVjZSBhIGZldyBiYXNpYyBleGFtcGxlcy4NCg0KIyMgQS4gVXNpbmcgYGBgcm5vcm0oKWBgYDogQmFzaWMgRXhhbXBsZXMgYW5kIHRoZSBgYGBzZXQuc2VlZCgpYGBgIEZ1bmN0aW9uLg0KDQpMZXQncyB1c2UgdGhlIHJub3JtIGZ1bmN0aW9uIHRvIHNpbXVsYXRlIDEwIG9ic2VydmF0aW9ucyBmcm9tIGEgcG9wdWxhdGlvbiB3aXRoIGEgbWVhbiBvZiAxNSBhbmQgYSBzdGFuZGFyZCBkZXZpYXRpb24gb2YgMi4gSSB3aWxsIHN0b3JlIHRoZXNlIDEwIGdlbmVyYXRlZCB2YWx1ZXMgaW4gYW4gb2JqZWN0IEkgY2FsbCAicm5vcm0uZXhhbXBsZSIgaW4gb3JkZXIgdG8gYmUgYWJsZSB0byBmaW5kIHRoZSBtZWFuIG9mIHRoZXNlIHZhbHVlcyAoYW5kIHBvdGVudGlhbGx5IHBlcmZvcm0gb3RoZXIgbWFuaXB1bGF0aW9ucyBhbmQgY2FsY3VsYXRpb25zIG9uIHRoZW0gbGF0ZXIgb24pLiANCg0KDQpgYGB7cn0NCnJub3JtLmV4YW1wbGU8LXJub3JtKDEwLCBtZWFuPTE1LCBzZD0yKQ0Kcm5vcm0uZXhhbXBsZQ0KbWVhbihybm9ybS5leGFtcGxlKQ0KYGBgDQpJdCBpcyBpbXBvcnRhbnQgdG8gbm90ZSB0aGF0IHdoaWxlIHRoZXNlIG9ic2VydmF0aW9ucyBhcmUgZHJhd24gZnJvbSBhIG5vcm1hbGx5IGRpc3RyaWJ1dGVkIHBvcHVsYXRpb24gd2l0aCBhIG1lYW4gb2YgMTUgYW5kIGEgc3RhbmRhcmQgZGV2aWF0aW9uIG9mIDIsIHRoZSBtZWFuIG9mIHRoaXMgc2FtcGxlIGlzIE5PVCAxNSwgYXMgc2VlbiBhYm92ZS4gIFRoZSBybm9ybSBmdW5jdGlvbiBkb2VzIG5vdCBjcmVhdGUgYSBub3JtYWxseSBkaXN0cmlidXRlZCBzYW1wbGUgd2l0aCBzcGVjaWZpZWQgbWVhbiwgc2QsIGFuZCBudW1iZXIgb2Ygb2JzZXJ2YXRpb25zIC0gaXQgZHJhd3Mgb2JzZXJ2YXRpb25zIGZyb20gYSB0aGVvcmV0aWNhbCBwb3B1bGF0aW9uIHdpdGggc3BlY2lmaWVkIHBhcmFtZXRlcnMuDQoNCg0KTm90ZSB0aGF0IHBlcmZvcm1pbmcgdGhlIHNhbWUgcm5vcm0gb3BlcmF0aW9uIHdpbGwgcmVzdWx0IGluIGEgZGlmZmVyZW50IHNldCBvZiB2YWx1ZXMgYmVpbmcgZ2VuZXJhdGVkIGVhY2ggdGltZToNCg0KYGBge3J9DQpybm9ybSgxMCwgbWVhbiA9IDE1LCBzZD0yKQ0Kcm5vcm0oMTAsIG1lYW4gPSAxNSwgc2Q9MikNCmBgYA0KDQpCeSBjb250cmFzdCwgcnVubmluZyB0aGUgbmFtZWQgb2JqZWN0ICJybm9ybS5leGFtcGxlIiByZXBlYXRlZGx5IHJldHVybnMgdGhlIHNhbWUgc2V0IG9mIHZhbHVlcyBlYWNoIHRpbWU6DQpgYGB7cn0NCnJub3JtLmV4YW1wbGUNCnJub3JtLmV4YW1wbGUNCmBgYA0KDQpIb3dldmVyLCByZS1ydW5uaW5nIHRoZSBvcmlnaW5hbCBleHByZXNzaW9uIGFuZCB0aGVuIGNhbGxpbmcgdGhlIG9iamVjdCB3aWxsIHJlc3VsdCBpbiB0aGUgdmFsdWVzIG9mICJybm9ybS5leGFtcGxlIiBiZWluZyBjaGFuZ2VkIGVhY2ggdGltZSB0aGUgZXhwcmVzc2lvbiBpcyByZS1ydW46DQoNCmBgYHtyfQ0Kcm5vcm0uZXhhbXBsZTwtcm5vcm0oMTAsIG1lYW49MTUsIHNkPTIpDQpybm9ybS5leGFtcGxlDQpybm9ybS5leGFtcGxlPC1ybm9ybSgxMCwgbWVhbj0xNSwgc2Q9MikNCnJub3JtLmV4YW1wbGUNCmBgYA0KDQpBcyBzdWNoLCBpZiB5b3Ugd2FudCB0byBvYnRhaW4gc2FtZSBzZXQgb2YgbnVtYmVycyBlYWNoIHRpbWUgeW91IHJ1biBhIGNodW5rIG9mIGNvZGUsIHlvdSBtdXN0IHVzZSB0aGUgc2V0LnNlZWQoKSBmdW5jdGlvbi4gIA0KDQpJZiB5b3Ugd2FudCB0byBnZW5lcmF0ZSB0aGUgc2FtZSBudW1iZXJzIGVhY2ggdGltZSB5b3UgcnVuIGFuIHJub3JtIGV4cHJlc3Npb24sIHlvdSBjYW4gdXNlIHRoZSBzZXQuc2VlZCBmdW5jdGlvbiwgd2hpY2ggdXNlcyB0aGUgZm9sbG93aW5nIHN5bnRheDoNCmBgYHNldC5zZWVkKHNlZWQpYGBgDQogICAgICANCkluIHdoaWNoICJzZWVkIiBpcyBhbiBpbnRlZ2VyIHRoYXQgc2V0cyB0aGUgc2VlZCwgc3VjaCB0aGF0IHNwZWNpZnlpbmcgdGhhdCBzYW1lIGludGVnZXIgbGF0ZXIgd2lsbCByZWNhbGwgdGhlIHBhcnRpY3VsYXIgc2V0IG9mIHZhbHVlcyBhc3NvY2lhdGVkIHdpdGggdGhhdCBzZWVkIGludGVnZXIuDQoNClRvIHNlZSBob3cgc2V0dGluZyBhIGdpdmVuIHNlZWQgd2lsbCBwcm9kdWNlIHRoZSBzYW1lIHZhbHVlcyBldmVyeSB0aW1lIHRoYXQgc2VlZCBpcyBzcGVjaWZpZWQsIGV2ZW4gaWYgdGhlIGV4cHJlc3Npb24gaXMgcmUtcnVuLCBsZXQncyByZXByb2R1Y2UgdGhlIHByZXZpb3VzIGNvZGUgY2h1bmsgYnV0IHNwZWNpZnkgYSBzZWVkIG9mICIxIiAgZm9yICJybm9ybS5leGFtcGxlIjoNCmBgYHtyfQ0Kc2V0LnNlZWQoMSkNCnJub3JtLmV4YW1wbGU8LXJub3JtKDEwLCBtZWFuPTE1LCBzZD0yKQ0Kcm5vcm0uZXhhbXBsZQ0KYGBgDQoNCmBgYHtyfQ0Kcm5vcm0uZXhhbXBsZTwtcm5vcm0oMTAsIG1lYW49MTUsIHNkPTIpDQpgYGANCg0KYGBge3J9DQpzZXQuc2VlZCgxKQ0Kcm5vcm0uZXhhbXBsZTwtcm5vcm0oMTAsIG1lYW49MTUsIHNkPTIpDQpybm9ybS5leGFtcGxlDQoNCmBgYA0KDQpVbmxpa2UgaW4gdGhlIHByZXZpb3VzIGNodW5rIG9mIGNvZGUsIHRoaXMgdGltZSByZS1ydW5uaW5nIHRoZSBleHByZXNzaW9uIGRvZXMgbm90IGNoYW5nZSB0aGUgbnVtYmVycyBpbiAicm5vcm0uZXhhbXBsZSIsIHNpbmNlIHdlIHNldCBhIHNlZWQuICBXaGF0IGhhcHBlbnMgd2hlbiB3ZSBzcGVjaWZ5IGEgc2VlZCBvdGhlciB0aGFuIDE/DQpgYGB7cn0NCnNldC5zZWVkKDIpDQpybm9ybS5leGFtcGxlPC1ybm9ybSgxMCwgbWVhbj0xNSwgc2Q9MikNCnJub3JtLmV4YW1wbGUNCmBgYA0KDQpgYGB7cn0NCnNldC5zZWVkKDIpDQpybm9ybS5leGFtcGxlPC1ybm9ybSgxMCwgbWVhbj0xNSwgc2Q9MikNCnJub3JtLmV4YW1wbGUNCmBgYA0KDQpTZXR0aW5nIGEgc2VlZCBvZiAyIHdpbGwgY3JlYXRlIGEgZGlmZmVyZW50IHNldCBvZiAxMCBnZW5lcmF0ZWQgdmFsdWVzLCB3aGljaCBjYW4gYmUgcmVwcm9kdWNlZCBlYWNoIHRpbWUgdGhlIHNlZWQgMiBpcyBzcGVjaWZpZWQuICBUaHVzLCAgZGlmZmVyZW50IHNlZWQgbnVtYmVycyBjYW4gYmUgdXNlZCB0byBzdG9yZSBkaWZmZXJlbnQgc2V0cyBvZiB2YWx1ZXMgZm9yIHRoZSBzYW1lIGV4cHJlc3Npb24uICBUaGUgc2VlZCBudW1iZXJzIEkgdXNlIHRocm91Z2hvdXQgdGhpcyBjaGFwdGVyIHdlcmUgY2hvc2VuIGJ5IG1lIGF0IHJhbmRvbSBhbmQgaG9sZCBubyBzeW1ib2xpYyBtZWFuaW5nIHdoYXRzb2V2ZXIuIEluIHRoZSBjYXNlIG9mIHRoaXMgY2hhcHRlciwgdGhlIHNldC5zZWVkKCkgZnVuY3Rpb24gaXMgdXNlZnVsIGFzIGl0IGVuc3VyZXMgdGhhdCB0aGF0LCB3aGVuZXZlciBJIHNwZWNpZnkgYSBzZWVkLCB0aGUgc2FtZSBudW1iZXJzIHdpbGwgYXBwZWFyIGVhY2ggdGltZSB0aGUgZnVsbCBjaGFwdGVyIGNvZGUgaXMgcnVuLg0KDQojIyBCLiBVc2luZyBybm9ybSBmb3IgQmFzaWMgU2ltdWxhdGlvbg0KDQpMZXQncyB1c2UgdGhlIHJub3JtIGZ1bmN0aW9uIHRvIHNpbXVsYXRlIHNvbWUgZGF0YSBpbiBhIHJlYWwtd29ybGQgY29udGV4dCEgDQoNClRvIHNldCB0aGUgc3RhZ2UgZm9yIHRoZSBzY2VuYXJpbyBJIHdhbnQgdG8gc2ltdWxhdGUsIEkgd2lsbCBmaXJzdCBuZWVkIHRvIGludHJvZHVjZSBhIG5ldyBjb25zdHJ1Y3Q6IA0KDQoiW1JFREFDVEVEXSwgaXQgd29uJ3Qgd29yayEiLiAgDQoNClRoaXMgY29uc3RydWN0IGRlc2NyaWJlcyB0aGUgbWl4dHVyZSBvZiBmcnVzdHJhdGlvbiwgZGVzcGFpciwgYW5kIHJlc2lnbmF0aW9uIHRoYXQgc3R1ZGVudHMgaW4gUiBjb3Vyc2VzIGV4cGVyaWVuY2Ugd2hlbiB0aGV5IGVuY291bnRlciBhIHBlcnBsZXhpbmcgcHJvYmxlbSBpbiBSLCB3aGV0aGVyIGl0IGJlIHBhY2thZ2VzIHRoYXQgcmVmdXNlIHRvIGluc3RhbGwsIGZ1bmN0aW9ucyB0aGF0IHlpZWxkIGluc2NydXRhYmxlIGVycm9yIG1lc3NhZ2VzLCBvciBjb2RlIHRoYXQgcnVucyBwZXJmZWN0bHkgb24gYSBuZWlnaGJvcmluZyBjb21wdXRlciBidXQgc3R1YmJvcm5seSByZWZ1c2VzIHRvIHJ1biBvbiB5b3Vycy4gV2UgY2FuIHVzZSB0aGUgcm5vcm0gZnVuY3Rpb24gdG8gc2ltdWxhdGUgdGhlIG51bWJlciBvZiBpbnN0YW5jZXMgb2YgdGhpcyBleHBlcmllbmNlIG92ZXIgdGhlIGNvdXJzZSBvZiBhIHNlbWVzdGVyLWxvbmcgUiBjbGFzcy4gIA0KDQpNb3JlIHNwZWNpZmljYWxseSwgd2Ugd2lsbCB1c2Ugcm5vcm0gdG8gIHNpbXVsYXRlIHRoZSBkYXRhIHdlIG1pZ2h0IGNvbGxlY3QgaW4gc3VydmV5aW5nIHRoZSBudW1iZXIgb2YgaW5zdGFuY2VzIG9mICJbUkVEQUNURURdLCBpdCB3b24ndCB3b3JrISIgdGhhdCBNYWMgYW5kIFBDIHVzZXJzIG1pZ2h0IGVuY291bnRlciBvdmVyIHRoZSBjb3Vyc2Ugb2YgdGhlIGZpcnN0IGFuZCBzZWNvbmQgaGFsZiBvZiBhIHNlbWVzdGVyLWxvbmcgUiBjb3Vyc2UuICBHZW5lcmFsbHksIHdoZW4gc2ltdWxhdGluZyBkYXRhLCB3ZSBlaXRoZXIgW1dIQVQgR09FUyBIRVJFPz8/IEEgV09SRCBPUiBQSFJBU0UgSVMgTUlTU0lOR10gb24ga25vd24gcG9wdWxhdGlvbiBwYXJhbWV0ZXJzIG9yIG9uIHRoZW9yZXRpY2FsbHkgZHJpdmVuIGFzc3VtcHRpb25zIHJlZ2FyZGluZyB0aGUgcG9wdWxhdGlvbiBwYXJhbWV0ZXJzLiBUaGUgZGF0YSBJIHdpbGwgZ2VuZXJhdGUgYXJlIGJhc2VkIG9uIHRoZSBmb2xsb3dpbmcgYXNzdW1wdGlvbnMgKGRlcml2ZWQgZnJvbSBpbmZvcm1hbCBvYnNlcnZhdGlvbiBvZiBteSBSIGNsYXNzKSByZWdhcmRpbmcgd2hhdCBvYnNlcnZhdGlvbnMgd291bGQgbG9vayBsaWtlOg0KDQoxLiAgSW5zdGFuY2VzIG9mICJbUkVEQUNURURdLCBpdCB3b24ndCB3b3JrISIgZW5jb3VudGVyZWQgZHVyaW5nIFIgdXNlIG1heSBiZSBjYXVzZWQgZWl0aGVyIGJ5IHVzZXIgZXJyb3IvdW5mYW1pbGlhcml0eSB3aXRoIHRoZSBSIHByb2dyYW0gb3IgYnkgUiBwcm9ncmFtIGJ1Z3MvY29tcGF0aWJpbGl0eSBpc3N1ZXMuDQoNCjIuICBBcyBzdHVkZW50cyBiZWNvbWUgbW9yZSBmYW1pbGlhciB3aXRoIHRoZSBSIHByb2dyYW0sIHVzZXIgZXJyb3IgZGVjcmVhc2VzLCBsZWFkaW5nIHRvIGZld2VyIGluc3RhbmNlcyBvZiAiW1JFREFDVEVEXSwgaXQgd29uJ3Qgd29yayEiIHN0ZW1taW5nIGZyb20gdXNlciBlcnJvciBmb3IgYm90aCBNYWMgYW5kIFBDIHVzZXJzIGluIDJuZCBoYWxmIG9mIHNlbWVzdGVyIGNvbXBhcmVkIHRvIDFzdCBoYWxmLg0KDQozLiBSIHN0dWRpbyBpcyBiZXR0ZXItb3B0aW1pemVkIGZvciBQQyB0aGFuIGZvciBNYWMsIGxlYWRpbmcgdG8gbW9yZSBpbnN0YW5jZXMgb2YgIltSRURBQ1RFRF0sIGl0IHdvbid0IHdvcmshIiBjYXVzZWQgYnkgcHJvZ3JhbSBlcnJvcnMgYW5kIGNvbXBhdGliaWxpdHkgaXNzdWVzIGZvciBNYWMgdXNlcnMgdGhhbiBmb3IgUEMgdXNlcnMuDQoNCg0KIEJhc2VkIG9uIHRoZXNlIGFzc3VtcHRpb25zLCBnaXZlbiBpbnN0YW5jZXMgb2YgIltSRURBQ1RFRF0sIGl0IHdvbid0IHdvcmshIiBhcyB0aGUgZGVwZW5kZW50IHZhcmlhYmxlLCBJIHdvdWxkIGV4cGVjdCB0byBvYnNlcnZlIGEgbWFpbiBlZmZlY3Qgb2YgVGltZSAobW9yZSBpbnN0YW5jZXMgaW4gMXN0IGhhbGYgdGhhbiAybmQgaGFsZiBvZiBzZW1lc3RlcikgYW5kIG9mIFVzZXIgVHlwZSAobW9yZSBpbnN0YW5jZXMgZm9yIE1hYyB1c2VycyB0aGFuIFBDIHVzZXJzKS4gIExldCdzIHVzZSB0aGUgcm5vcm0gZnVuY3Rpb24gdG8gc2ltdWxhdGUgdGhpcyBwYXR0ZXJuIG9mIHJlc3VsdHMuIE5vdGUgdGhhdCBvdGhlciBmYWN0b3JzIHRoYXQgbWlnaHQgaW5mbHVlbmNlIHRoZSBkZXBlbmRlbnQgdmFyaWFibGUsIHN1Y2ggYXMgaW5kaXZpZHVhbCBSIHByb2ZpY2llbmN5IGFuZCBwc3ljaG9sb2dpY2FsIGVmZmVjdHMgb2YgUiBjb3Vyc2UgcHJvZmVzc29ycyBmcmVxdWVudGx5IGRpc3BhcmFnaW5nIE1hY3MsIGFyZSBub3QgbW9kZWxlZCBoZXJlLg0KYGBge3J9DQpzZXQuc2VlZCgxMDMpDQpNYWNVc2VyczFzdEhhbGYgPC0gcm5vcm0oMTAwLCBtZWFuID0gNTAwLCBzZCA9IDM1KQ0KUENVc2VyczFzdEhhbGYgPC0gcm5vcm0oMTAwLCBtZWFuID0gNDUwLCBzZCA9IDM1KQ0KTWFjVXNlcnMybmRIYWxmIDwtcm5vcm0oMTAwLCBtZWFuID0gMTUwLCBzZCA9IDM1KQ0KUENVc2VyczJuZEhhbGYgPC0gcm5vcm0gKDEwMCwgbWVhbiA9IDEwMCwgc2Q9MzUpDQpgYGANCg0KDQpBcyB5b3Ugc2VlIGluIHRoZSBzY3JpcHQgYWJvdmUsIEkgc2V0IHRoZSBhdmVyYWdlcyB0byBzaW11bGF0ZSB0aGUgaHlwb3RoZXNpemVkIHBhdHRlcm4gb2YgcmVzdWx0cy4gIEFzIEkgaGFkIG5vIHByZWRpY3Rpb25zIHJlZ2FyZGluZyB0aGUgZGlzdHJpYnV0aW9uIG9mIGRhdGEsIEkgaGF2ZSBubyBzdHJvbmcgdGhlb3JldGljYWwgcmF0aW9uYWxlIGZvciBzZXR0aW5nIGEgc3RhbmRhcmQgZGV2aWF0aW9uIG9mIDM1LiBIb3dldmVyLCBrZWVwaW5nIHRoZSBkZWZhdWx0IHN0YW5kYXJkIGRldmlhdGlvbiB2YWx1ZSBvZiAxIHdvdWxkIGhhdmUgeWllbGRlZCB1bm5hdHVyYWwtbG9va2luZyBzaW11bGF0ZWQgZGF0YSwgYXMgdGhlIGRpc3RyaWJ1dGlvbiBvZiBkYXRhIHdvdWxkIGhhdmUgYmVlbiB0b28gdGlnaHRseSBjbHVzdGVyZWQgYXJvdW5kIHRoZSBtZWFuLiAgSSB1c2VkIHRoZSBzZXQuc2VlZCBmdW5jdGlvbiB0byBlbnN1cmUgdGhhdCB0aGUgc2FtZSB2YWx1ZXMgd2lsbCBiZSBnZW5lcmF0ZWQgZWFjaCB0aW1lIHRoaXMgY29kZSBpcyBydW4uDQoqKkFnYWluLCB3aHkgbm90IGp1c3QgdXNlIHRoZSBvYmplY3QgbGlrZSB5b3UgZGlkIHdpdGggcm5vcm0uZXhhbXBsZT8gV2h5IGFsd2F5cyB1c2Ugc2V0LnNlZWQ/IElmIHlvdSBnaXZlIHRoZSBleHBsYW5hdGlvbiBiZWZvcmUgdGhpcyBtZW50aW9uIG9mIGl0LCBJIHdvdWxkbid0IGJlIHF1ZXN0aW9uaW5nIGl0IGRvd24gaGVyZS4qKg0KVGhlIG5leHQgc2VjdGlvbiBkaXNjdXNzZXMgdGhlIHJlcCBmdW5jdGlvbi4gIE9uY2Ugd2UgaGF2ZSBsZWFybmVkIHRoZSBiYXNpY3Mgb2YgdGhlIHJlcCBmdW5jdGlvbiwgd2Ugd2lsbCBiZSBhYmxlIHRvIGRvIG1vcmUgd2l0aCB0aGUgb2JzZXJ2YXRpb25zIHdlIGhhdmUgc2ltdWxhdGVkLg0KDQojIFRoZSByZXAgRnVuY3Rpb24gYW5kIGl0cyBBcHBsaWNhdGlvbnMNCiANCkluIHRoaXMgc2VjdGlvbiwgSSB3aWxsIGludHJvZHVjZSB0aGUgcmVwIGZ1bmN0aW9uIGFuZCBwcm92aWRlIGEgYmFzaWMgdXNlIGNhc2UgaW4gdGhlIGNvbnRleHQgb2YgY3JlYXRpbmcgYSBkYXRhIGZyYW1lIGZyb20gdGhlIHNpbXVsYXRlZCBkYXRhIHdlIGhhdmUgZ2VuZXJhdGVkLiAgVGhlIHJlcCBmdW5jdGlvbiByZXBsaWNhdGVzIHRoZSB2YWx1ZXMgaW4gYSBnaXZlbiBvYmplY3QgKGEgdmVjdG9yIG9yIGEgZnVuY3Rpb24pIGEgc3BlY2lmaWVkIG51bWJlciBvZiB0aW1lcy4gIFRoZSByZXAgZnVuY3Rpb24gdXNlcyB0aGUgZm9sbG93aW5nIHN5bnRheCBhbmQgYXJndW1lbnRzOg0KICAgICAgICAgICAgDQogICAgICAgICAgICBgYGByZXAoeCwgLi4uKWBgYA0KICAgICAgICAgICAgDQpXaGVyZSAieCIgaXMgdGhlIHZhbHVlIHRvIGJlIHJlcGxpY2F0ZWQgb3IgdGhlIHZlY3RvciBjb250YWluaW5nIHZhbHVlcyB0byBiZSByZXBsaWNhdGVkLCBhbmQgIi4uLiIgZGVub3RlcyBhZGRpdGlvbmFsIGFyZ3VtZW50cywgc3VjaCBhcyAidGltZXM9IiAod2hpY2ggc3BlY2lmaWVzIHRoZSBudW1iZXIgb2YgdGltZXMgdG8gcmVwbGljYXRlICJ4IiksICJlYWNoPSIgKHdoaWNoIHNwZWNpZmllcyB0aGUgbnVtYmVyIG9mIHRpbWVzIHRvIHJlcGVhdCBlYWNoIGVsZW1lbnQgb2YgIngiKSwgYW5kICJsZW49IiAod2hpY2ggc3BlY2lmaWVzIHRoZSBkZXNpcmVkIGxlbmd0aCBvZiBhbiBvdXRwdXQgdmVjdG9yKS4gIEJlbG93LCBJIGdvIG92ZXIgc29tZSB2ZXJ5IGJhc2ljIGV4YW1wbGVzIG9mIGVhY2ggb2YgdGhlc2UgdXNlcyBvZiB0aGUgcmVwIGZ1bmN0aW9uLg0KDQojIyBBLiBFeGFtcGxlcyBvZiBzaW1wbGUgcmVwbGljYXRpb24gb2YgaW5kaXZpZHVhbCB2YWx1ZXMgYW5kIHZlY3RvcnMNCiMjIyMgVGhlIHNpbXBsZXN0IHVzZSBvZiB0aGUgcmVwbGljYXRlIGZ1bmN0aW9uIGlzIHRvIHJlcGxpY2F0ZSBhIHRleHQgb3IgbnVtZXJpYyB2YWx1ZSBhIHNldCBudW1iZXIgb2YgdGltZXMuICBUbyBiZWdpbiwgbGV0J3MgcmVwbGljYXRlIHRoZSBudW1iZXIgNSB0ZW4gdGltZXM6DQpgYGB7cn0NCnJlcCg1LCAxMCkNCmBgYA0KQXMgeW91IHNlZSBhYm92ZSwgZW50ZXJpbmcgYSB2YWx1ZSBuIHN1Y2ggdGhhdCB0aGUgc3l0YXggcmVhZHMgInJlcCh4LG4pIiBkZWZhdWx0cyB0byB0aGUgc2FtZSBtZWFuaW5nIGFzIHRoZSBzeW50YXggInJlcCh4LCB0aW1lcz1uKSIuICBBc2lkZSBmcm9tIGluZGl2aWR1YWwgdmFsdWVzLCB3ZSBjYW4gYWxzbyByZXBsaWNhdGUgYSB0ZXh0IGxhYmVsIGluIHRoZSBzYW1lIG1hbm5lcjoNCmBgYHtyfQ0KcmVwKCJLaXR0eSBDYXQiLCAxMCkNCmBgYA0KT25lIHBvc3NpYmxlIHVzZSBmb3IgcmVwbGljYXRpbmcgaW5kaXZpZHVhbCB2YWx1ZXMgb3IgdGV4dCBsYWJlbHMgaXMgcG9wdWxhdGluZyBjb2x1bW5zIGZvciBjb250aW51b3VzIG9yIGNhdGVnb3JpY2FsIGRhdGEgaW4gYSBkYXRhIGZyYW1lLiBJIGV4cGxvcmUgdGhpcyB1c2UgaW4gc2VjdGlvbiA1LjIgQiBiZWxvdy4NCg0KQXNpZGUgZnJvbSByZXBsaWNhdGluZyBpbmRpdmlkdWFsIHRleHQgb3IgbnVtZXJpYyB2YWx1ZXMsIHRoZSByZXAgZnVuY3Rpb24gaXMgY29tbW9ubHkgdXNlZCB0byByZXBsaWNhdGUgKGVsZW1lbnRzIHdpdGhpbikgdmVjdG9ycy4gIEJlbG93LCBJIGNyZWF0ZSBhIHNpbXBsZSB2ZWN0b3Igb2Ygc2V2ZW4gbnVtZXJpYyB2YWx1ZXMgYW5kIHRoZW4gcmVwbGljYXRlIHRoYXQgdmVjdG9yLCBhcyBhIHdob2xlLCAzIHRpbWVzLg0KYGBge3J9DQpFeGFtcGxlUmVwVmVjdG9yPC1jKDEsMiw1LDcsNyw4LDEwKQ0KcmVwKEV4YW1wbGVSZXBWZWN0b3IsIDMpDQpgYGANCldlIGNhbiBhbHNvIHJlcGxpY2F0ZSBlYWNoIGVsZW1lbnQgb2YgdGhpcyB2ZWN0b3IgYSBzZXQgbnVtYmVyIG9mIHRpbWVzLiAgQmVsb3csIEkgZmlyc3QgcmVwbGljYXRlIGVhY2ggZWxlbWVudCBvZiB0aGUgdmVjdG9yIHR3aWNlLiAgTmV4dCwgSSByZXBsaWNhdGUgZWFjaCBlbGVtZW50IHR3aWNlLCB3aGlsZSBhbHNvIHJlcGxpY2F0aW5nIHRoZSB2ZWN0b3IgYXMgYSB3aG9sZSAod2l0aCBlYWNoIGVsZW1lbnQgcHJlc2VudCB0d2ljZSkgdGhyZWUgdGltZXMuDQoNCmBgYHtyfQ0KcmVwKEV4YW1wbGVSZXBWZWN0b3IsIGVhY2g9MikNCnJlcChFeGFtcGxlUmVwVmVjdG9yLCB0aW1lcz0zLCBlYWNoPTIpDQpgYGANCkZpbmFsbHksIGxldCdzIHNlZSB3aGF0IGhhcHBlbnMgd2hlbiB3ZSBzcGVjaWZ5IGFuIG91dHB1dCB2ZWN0b3Igb2YgYSBkaWZmZXJlbnQgbGVuZ3RoIHRoYW4gdGhlIG51bWJlciBvZiB0ZXJtcyBpbiB0aGUgb3JpZ2luYWwgdmVjdG9yICg3KToNCg0KMjAgdGVybXM6DQpgYGB7cn0NCnJlcChFeGFtcGxlUmVwVmVjdG9yLCBsZW49MjApDQpgYGANCjUgdGVybXM6DQpgYGB7cn0NCnJlcChFeGFtcGxlUmVwVmVjdG9yLCBsZW49NSkNCmBgYA0KV2hlbiBvdXRwdXR0aW5nIHRvIGEgdmVjdG9yIGxvbmdlciB0aGFuIHRoZSBvcmlnaW5hbCB2ZWN0b3IsIHRoZSBvcmlnaW5hbCB2ZWN0b3IgaXMgcmVwbGljYXRlZCB1bnRpbCB0aGUgb3V0cHV0IHZlY3RvciBoYXMgYmVlbiBmaWxsZWQuIEluIG91ciBjYXNlLCBmaWxsaW5nIHRoZSAyMCB0ZXJtIG91dHB1dCB2ZWN0b3IgcmVxdWlyZXMgdGhlIG9yaWdpbmFsIHZlY3RvciB0byBiZSByZXBsaWNhdGVkIHR3aWNlLCB3aXRoIGFuIGluY29tcGxldGUgM3JkIHJlcGxpY2F0aW9uIG9mIG9ubHkgdGhlIGZpcnN0IHNpeCB0ZXJtcy4NCg0KV2hlbiBvdXRwdXR0aW5nIHRvIGEgdmVjdG9yIHNob3J0ZXIgdGhhbiB0aGUgb3JpZ2luYWwgdmVjdG9yLCB0aGUgb3V0cHV0IHZlY3RvciBpcyBmaWxsZWQgYmVmb3JlIGFsbCBvZiB0aGUgdmFsdWVzIG9mIHRoZSBvcmlnaW5hbCB2ZWN0b3IgY2FuIGJlIHJlcGxpY2F0ZWQuIEluIG91ciBjYXNlLCB0aGUgb3JpZ2luYWwgdmVjdG9yIGhhcyA3IHZhbHVlcyBhbmQgdGhlIG91dHB1dCB2ZWN0b3Igb25seSBoYXMgcm9vbSBmb3IgNSwgc28gdGhlIGxhc3QgMiB2YWx1ZXMgYXJlIG5vdCByZXBsaWNhdGVkLg0KDQojIyBCLiBVc2luZyB0aGUgcmVwIEZ1bmN0aW9uIHRvIENyZWF0ZSBEYXRhIEZyYW1lcw0KIyMjI05vdyB0aGF0IHdlJ3ZlIGdvbmUgb3ZlciB0aGUgYmFzaWMgdXNlcyBvZiB0aGUgcmVwIGZ1bmN0aW9uLCBsZXQncyB1c2UgdGhlIHJlcCBmdW5jdGlvbiB0byBoZWxwIHVzIGNyZWF0ZSBhIGRhdGEgZnJhbWUgZm9yIHRoZSBzaW11bGF0ZWQgZGF0YSB3ZSBjcmVhdGVkIGluIHNlY3Rpb24gNS4xIC4gIFJlY2FsbCB0aGF0IHdlIGhhdmUgb25lIGJldHdlZW4tc3ViamVjdHMgdmFyaWFibGUsIFVzZXJUeXBlIChNYWMgdXNlciBvciBQQyB1c2VyKSwgYW5kIG9uZSB3aXRoaW4tc3ViamVjdHMgdmFyaWFibGUsIFRpbWUgKDFzdCBhbmQgMm5kIEhhbGYgb2YgU2VtZXN0ZXIpLiAgQXMgd2UgaGF2ZSAxMDAgTWFjIFVzZXJzIGFuZCAxMDAgUEMgdXNlcnMsIG91ciB0b3RhbCBudW1iZXIgb2Ygc3ViamVjdHMgaXMgMjAwLiAgDQoNCkluIGNyZWF0aW5nIHRoZSBkYXRhIGZyYW1lLCBJIHdpbGwgdXNlIHRoZSByZXAgZnVuY3Rpb24gdG8gcmVwbGljYXRlIHRoZSBsYWJlbHMgb2YgZWFjaCBsZXZlbCBvZiBlYWNoIElWIGFjcm9zcyB0aGUgcmVsZXZhbnQgcmFuZ2Ugb2Ygc3ViamVjdHMuICBCZWxvdywgSSBzZXQgdGhlc2UgYXJndW1lbnRzIGFuZCBjcmVhdGUgdGhlIGRhdGEgZnJhbWU6DQoNCg0KYGBge3J9DQpuLk1hY1VzZXJzPTEwMA0Kbi5QQ1VzZXJzPTEwMA0KTj1uLk1hY1VzZXJzICsgbi5QQ1VzZXJzDQoNClNpbXVsYXRlZERhdGEgPC0gZGF0YS5mcmFtZSgNCiAgU3ViamVjdCA9IGMoc2VxKDEsTiksc2VxKDEsTikpLA0KICBUaW1lID0gYyhyZXAoIjFzdCBIYWxmIiwgTiksIHJlcCgiMm5kIEhhbGYiLCBOKSksDQogIFVzZXJUeXBlID0gYyhyZXAoYyhyZXAoIk1hYyBVc2VyIiwgbi5NYWNVc2VycyksIHJlcCgiUEMgVXNlciIsIG4uUENVc2VycykpLDIpKSwNCiAgSW5zdGFuY2VzID0gYyhNYWNVc2VyczFzdEhhbGYsUENVc2VyczFzdEhhbGYsIA0KICAgICAgICAgICAgICAgIE1hY1VzZXJzMm5kSGFsZixQQ1VzZXJzMm5kSGFsZg0KICAgICAgICAgICANCiAgKQ0KKQ0KYGBgDQoNCg0KVXNpbmcgdGhlIHJlcCBmdW5jdGlvbiB0byBkZW5vdGUgd2hpY2ggbGV2ZWwgb2YgdGhlIHdpdGhpbi0gYW5kIGJldHdlZW4tc3ViamVjdHMgSVYgZWFjaCBkYXRhIHBvaW50IGlzIGFzc29jaWF0ZWQgd2l0aCwgYXMgaW4gdGhlIGFib3ZlIGRhdGEgZnJhbWUgc3ludGF4LCBhbGxvd3MgdXMgdG8gIHNlZSB3aGljaCB2YWx1ZXMgY29ycmVzcG9uZCB0byBNYWMgVXNlcnMgYW5kIFBDIFVzZXJzLCBhbmQgdG8gdGhlIDFzdCBIYWxmIGFuZCAybmQgSGFsZiBvZiB0aGUgc2VtZXN0ZXIuIExldCdzIG1ha2UgYSB0YWJsZSBvZiB0aGVzZSBkYXRhLiAgQXMgdGhlcmUgYXJlIGEgdG90YWwgb2YgNDAwIGNvbHVtbnMgaW4gdGhlIGZ1bGwgdGFibGUsIEkgd2lsbCBiZSB1c2luZyB0aGUgaGVhZCgpIGZ1bmN0aW9uIHRvIGRpc3BsYXkgb25seSB0aGUgZmlyc3Qgc2V2ZXJhbCBjb2x1bW5zLiAgSSB3aWxsIGJlIHVzaW5nIHRoZSBrYWJsZSBmdW5jdGlvbiBpbiB0aGUga25pdHIgcGFja2FnZSB0byBwcm9kdWNlIHRoaXMgdGFibGU7IGZ1cnRoZXIgZG9jdW1lbnRhdGlvbiBhbmQgaGVscCBmb3IgdGhpcyBwYWNrYWdlIGNhbiBiZSBmb3VuZCBhdCB0aGUgZm9sbG93aW5nIFVSTDogaHR0cDovL3lpaHVpLm5hbWUva25pdHIvDQoNCmBgYHtyLCByZXN1bHRzPSdhc2lzJ30NCmtuaXRyOjprYWJsZShoZWFkKFNpbXVsYXRlZERhdGEpKQ0KYGBgDQoNClRoaXMgdGFibGUgY291bGQgYmUgbW9yZSBlYXNpbHkgaW50ZXJwcmV0ZWQgaWYgdGhlIHR3byBsZXZlbHMgb2YgVGltZSwgMXN0IEhhbGYgYW5kIDJuZCBIYWxmLCB3ZXJlIHJlcHJlc2VudGVkIG9uIHNlcGFyYXRlIGNvbHVtbnMuICBXZSBjYW4gdXNlIHRoZSBkY2FzdCgpIGZ1bmN0aW9uIChwYXJ0IG9mIHRoZSAicmVzaGFwZTIiIHBhY2thZ2UpIHRvIGNoYW5nZSB0aGUgc3RydWN0dXJlIG9mIHRoZSBkYXRhIGZyYW1lIGFuZCB0aGVuIGNyZWF0ZSBhIG5ldyB0YWJsZS4gQmVsb3csIEkgdXNlIHRoZSBkY2FzdCBmdW5jdGlvbiB0byByZXN0cnVjdHVyZSB0aGUgZGF0YSBmcmFtZSBieSBzcGxpdHRpbmcgdGhlIHZhcmlhYmxlIHRpbWUgaW50byB0d28gY29sdW1ucyBjb3JyZXNwb25kaW5nIHRvIHRoZSAyIGxldmVscyBvZiB0aGUgdmFyaWFibGU6IEZpcnN0IEhhbGYgYW5kIFNlY29uZCBIYWxmLiBBZ2FpbiwgSSB1c2UgdGhlIGhlYWQoKSBmdW5jdGlvbiB0byBkaXNwbGF5IG9ubHkgdGhlIGZpcnN0IHNldmVyYWwgY29sdW1ucyBvZiBhIHRhYmxlIHRoYXQgaGFzIDIwMCBjb2x1bXNuIGluIHRvdGFsLiAgRnVydGhlciBkb2N1bWVudGF0aW9uIGFuZCBoZWxwIHdpdGggdGhlIHJlc2hhcGUgMiBwYWNrYWdlIGNhbiBiZSBmb3VuZCBhdCB0aGUgZm9sbG93aW5nIFVSTDogaHR0cDovL3NlYW5hbmRlcnNvbi5jYS8yMDEzLzEwLzE5L3Jlc2hhcGUuaHRtbA0KDQpgYGB7cn0NClNpbXVsYXRlZERhdGEuV2lkZTwtZGNhc3QoU2ltdWxhdGVkRGF0YSwgU3ViamVjdCArIFVzZXJUeXBlIH4gVGltZSwgdmFsdWUudmFyPSJJbnN0YW5jZXMiKQ0Ka25pdHI6OmthYmxlKGhlYWQoU2ltdWxhdGVkRGF0YS5XaWRlKSkNCg0KYGBgDQoNCkFzIHlvdSBjYW4gc2VlLCB0aGUgdmFyaWFibGUgIlRpbWUiIGhhcyBub3cgYmVlbiBzcGxpdCBpbnRvIHR3byBjb2x1bW5zLCAiMXN0IEhhbGYiIGFuZCAiMm5kIEhhbGYiLiAgVG8gc2VlIHRoZSBmdWxsIHRhYmxlLCBlaXRoZXIgd2l0aCBvciB3aXRob3V0IHRoZSBzcGxpdCwgdmlldyB0aGUgb2JqZWN0IGluIHRoZSBHbG9iYWwgRW52aXJvbm1lbnQuDQoNCg0KIyBUaGUgc2FtcGxlIEZ1bmN0aW9uDQoNCkluIHRoaXMgc2VjdGlvbiwgSSB3aWxsIGludHJvZHVjZSB0aGUgc2FtcGxlIGZ1bmN0aW9uLCBwcm92aWRlIGV4YW1wbGVzIG9mIGJhc2ljIHVzZSwgYW5kIGRpc2N1c3Mgc2FtcGxpbmcgd2l0aCBhbmQgd2l0aG91dCByZXBsYWNlbWVudCBhbmQgYm9vdHN0cmFwcGluZy4gIFRoZSBzYW1wbGUgZnVuY3Rpb24gaXMgbW9zdCBjb21tb25seSB1c2VkIHRvIHRha2UgYSBzYW1wbGUgb2YgdGhlIGVsZW1lbnRzIG9mIGEgdmVjdG9yLCB3aGljaCBjYW4gYmUgZG9uZSBlaXRoZXIgd2l0aCBvciB3aXRob3V0IHJlcGxhY2VtZW50LiAgVGhlIHNhbXBsZSBmdW5jdGlvbiB1c2VzIHRoZSBmb2xsb3dpbmcgc3ludGF4IGFuZCBhcmd1bWVudHM6DQoNCmBgYHNhbXBsZSh4LCBzaXplLCByZXBsYWNlID0gRkFMU0UpYGBgDQoNCldoZXJlICJ4IiBpcyBhIHZlY3RvciBvZiBlbGVtZW50cyB0byBjaG9vc2UgZnJvbSwgInNpemUiIGlzIHRoZSBudW1iZXIgb2YgZWxlbWVudHMgdG8gYmUgc2FtcGxlZCwgYW5kICJyZXBsYWNlPSIgY2FuIGJlIHNldCB0byBlaXRoZXIgIlRSVUUiIG9yICJGQUxTRSIgZGVwZW5kaW5nIG9uIHdoZXRoZXIgeW91IHdpc2ggdG8gc2FtcGxlIHdpdGggb3Igd2l0aG91dCByZXBsYWNlbWVudC4gIFNwZWNpZnlpbmcgcmVwbGFjZSA9ICJUUlVFIiB3aWxsIHNhbWxlIHdpdGggcmVwbGFjZW1lbnQsIHdoZXJlYXMgc3BlY2lmeWluZyByZXBsYWNlID0gIkZBTFNFIiB3aWxsIHNhbXBsZSB3aXRob3V0IHJlcGxhY2VtZW50LiAgQmVsb3csIEkgZ28gb3ZlciBiYXNpYyB1c2VzIG9mIHRoZSAic2FtcGxlIiBmdW5jdGlvbiB0byBzYW1wbGUgd2l0aCBhbmQgd2l0aG91dCByZXBsYWNlbWVudC4NCg0KIyMgQS4gQmFzaWMgRXhhbXBsZXMgVXNpbmcgdGhlIHNhbXBsZSBGdW5jdGlvbg0KDQpGaXJzdCwgbGV0J3MgY3JlYXRlIGEgdmVjdG9yIG9mIHZhbHVlcyB3aGljaCBkb2VzIG5vdCBpbmNsdWRlIGFueSByZXBlYXRlZCBlbGVtZW50cy4gIFRoaXMgd2lsbCBtYWtlIGl0IGVhc2llciB0byBzZWUgdGhlIGRpZmZlcmVuY2UgYmV0d2VlbiBzYW1wbGluZyB3aXRoIGFuZCB3aXRob3V0IHJlcGxhY2VtZW50Lg0KYGBge3J9DQpFeGFtcGxlU2FtcGxlVmVjdG9yPC1jKDEsMiwzLDQsNSw2LDcsOCw5LDEwLDExLDEyLDEzLDE0LDE1LDE2LDE3LDE4LDE5LDIwKQ0KYGBgDQpMZXQncyBzYW1wbGUgZmlmdGVlbiB2YWx1ZXMgZnJvbSB0aGlzIHZlY3RvciBXSVRIT1VUIHJlcGxhY2VtZW50DQpgYGB7cn0NCnNhbXBsZShFeGFtcGxlU2FtcGxlVmVjdG9yLDE1LHJlcGxhY2UgPSBGQUxTRSkNCmBgYA0KQXMgeW91IGNhbiBzZWUsIGp1c3QgYXMgbm8gdmFsdWUgYXBwZWFyZWQgaW4gdGhlIG9yaWdpbmFsIHZlY3RvciBtb3JlIHRoYW4gb25jZSwgbm8gdmFsdWUgYXBwZWFycyBpbiBvdXIgc2FtcGxlIG1vcmUgdGhhbiBvbmNlLiBTYW1wbGluZyB3aXRob3V0IHJlcGxhY2VtZW50IG1lYW5zIHRoYXQgb25jZSBhIHZhbHVlIGhhcyBiZWVuIHNhbXBsZWQsIGl0IGlzICJyZW1vdmVkIiBmcm9tIHRoZSBvcmlnaW5hbCB2ZWN0b3IgYW5kIGNhbm5vdCBiZSByYW5kb21seSBzYW1wbGVkIGFnYWluLg0KDQpOb3csIGxldCdzIHNhbXBsZSBmaWZ0ZWVuIHZhbHVlcyBmcm9tIHRoaXMgdmVjdG9yIFdJVEggcmVwbGFjZW1lbnQNCmBgYHtyfQ0Kc2FtcGxlKEV4YW1wbGVTYW1wbGVWZWN0b3IsMTUscmVwbGFjZSA9IFRSVUUpDQpgYGANClRoaXMgdGltZSwgaXQgaXMgbGlrZWx5IHRoYXQgdGhlcmUgaXMgYXQgbGVhc3Qgb25lIGR1cGxpY2F0ZSB2YWx1ZSBpbiBvdXIgc2FtcGxlLiAgU2FtcGxpbmcgd2l0aCByZXBsYWNlbWVudCBtZWFucyB0aGF0IHNhbXBsZWQgdmFsdWVzIGFyZSAicmVwbGFjZWQiIGluIHRoZSBvcmlnaW5hbCB2ZWN0b3IgYW5kIHRodXMgbWF5IGJlIHJhbmRvbWx5IHNhbXBsZWQgYWdhaW4uDQoNCldoZW4gZHJhd2luZyByYW5kb20gc2FtcGxlcyBvZiBhIGZldyBvYnNlcnZhdGlvbnMsIGEgZGlmZmVyZW50IHNldCBvZiBvYnNlcnZhdGlvbnMgd2lsbCBnZW5lcmFsbHkgYmUgc2FtcGxlZCBlYWNoIHRpbWUgeW91IHJ1biB0aGUgc2FtcGxlIGZ1bmN0aW9uLiAgSWYgeW91IHdhbnQgdG8ga2VlcCB0aGUgc2FtZSBzZXQgb2Ygc2FtcGxlZCBvYnNlcnZhdGlvbnMgaW4gb3JkZXIgdG8gcGVyZm9ybSBmdXJ0aGVyIG1hbmlwdWxhdGlvbnMgdXNpbmcgdGhhdCBwYXJ0aWN1bGFyIHNldCwgeW91IGNhbiB1c2UgdGhlIHNldC5zZWVkIGZ1bmN0aW9uIGhlcmUgYXMgd2VsbDoNCg0KYGBge3J9DQpzZXQuc2VlZCgyKTsgc2FtcGxlKEV4YW1wbGVTYW1wbGVWZWN0b3IsIDE1KQ0Kc2V0LnNlZWQoMik7IHNhbXBsZShFeGFtcGxlU2FtcGxlVmVjdG9yLCAxNSkNCmBgYA0KQXMgeW91IGNhbiBzZWUsIHRoZSB0d28gc2FtcGxlZCBzZXRzIGFyZSBpZGVudGljYWwsIHRoYW5rcyB0byB0aGUgc2V0LnNlZWQgZnVuY3Rpb24uIEkgY2FuIHVzZSB0aGlzIHNlZWQgdG8gcmVjYWxsIHRoaXMgcGFydGljdWxhciBzZXQgb2YgZmlmdGVlbiBzYW1wbGVkIHZhbHVlcyBpZiBJIHdhbnQgdG8gdXNlIHRoZW0gaW4gdGhlIGZ1dHVyZS4NCg0KDQojIyBCLiBVc2luZyB0aGUgc2FtcGxlIEZ1bmN0aW9uIGFuZCBCb290c3RyYXBwaW5nIG9uIG91ciBTaW11bGF0ZWQgTWFjIGFuZCBQQyBVc2VyIERhdGENCg0KTm93LCB3ZSB3aWxsIGRpc2N1c3Mgc2FtcGxpbmcgaW4gdGhlIGNvbnRleHQgb2Ygb3VyIHNpbXVsYXRlZCBkYXRhLCBhbmQgaW50cm9kdWNlIHRoZSBjb25jZXB0IG9mIGJvb3RzdHJhcHBpbmcuICBCb290c3RyYXBwaW5nIGludm9sdmVzIHJlc2FtcGxpbmcgZnJvbSBhIHBvcHVsYXRpb24gb3Igc2V0IG9mIHZhbHVlcyBhIHNwZWNpZmllZCBudW1iZXIgb2YgdGltZXMgaW4gb3JkZXIgZm9yIHRoZSBtZWFuIG9mIHlvdXIgc2FtcGxlcyB0byBtb3JlIGNsb3NlbHkgYXBwcm94aW1hdGUgdGhlIG1lYW4gb2YgdGhlIHBvcHVsYXRpb24gb3Igb3JpZ2luYWwgc2V0IG9mIHZhbHVlcy4gIFdoaWxlIHRoZXJlIGFyZSBzZXZlcmFsIHdheXMgdG8gcGVyZm9ybSBib290c3RyYXBwZWQgc2FtcGxpbmcgaW4gUiwgdGhlIHNpbXBsZXN0IGlzIHRvIHVzZSB0aGUgcmVwbGljYXRlIGZ1bmN0aW9uLCB3aGljaCBJIHdpbGwgZGlzY3VzcyBvbmx5IGFzIGl0IHBlcnRhaW5zIHRvIGJvb3RyYXBwaW5nLiAgVGhlIHJlcGxpY2F0ZSBmdW5jdGlvbiByZXBlYXRzIGEgZ2l2ZW4gZnVuY3Rpb24gYSBzcGVjaWZpZWQgbnVtYmVyIG9mIHRpbWVzLCB1c2luZyB0aGUgZm9sbG93aW5nIHN5bnRheCBhbmQgYXJndW1lbnRzOg0KDQpgYGByZXBsaWNhdGUobiwgZXhwcilgYGANCg0KV2hlcmUgbiBpcyB0aGUgbnVtYmVyIG9mIGRlc2lyZWQgcmVwbGljYXRpb25zIGFuZCAiZXhwciIgaXMgdGhlIGV4cHJlc3Npb24gdG8gYmUgcmVwbGljYXRlZCwgdXN1YWxseSBhbiBvYmplY3QgYWxyZWFkeSBlbnRlcmVkIGluIHRoZSBnbG9iYWwgZW52aXJvbm1lbnQuDQoNCkJvb3N0cmFwcGluZyBpcyBvZnRlbiB1c2VkIHdoZW4gc2FtcGxpbmcgZnJvbSBhIG5vcm1hbGx5IGRpc3RyaWJ1dGVkIHBvcHVsYXRpb24uICBIZXJlLCBJIGRpc2N1c3Mgc2FtcGxpbmcgYW5kIGJvb3RzdHJhcHBpbmcgc2FtcGxlcyBmcm9tIGEgbGFyZ2VyIHNldCBvZiBvYnNlcnZhdGlvbnMuICBJbiB0aGUgbmV4dCBzZWN0aW9uLCA1LjQsIEkgZGlzY3VzcyBzYW1wbGluZyBmcm9tIGEgcG9wdWxhdGlvbiBpbiB0aGUgY29udGV4dCBvZiBkZW1vbnN0cmF0aW5nIHRoZSBMYXcgb2YgTGFyZ2UgTnVtYmVycy4NCg0KTGV0J3Mgc2FtcGxlIHRlbiB2YWx1ZXMgZnJvbSBvdXIgZGF0YSBjb3JyZXNwb25kaW5nIHRvIE1hYyBVc2VycyBpbiB0aGUgZmlyc3QgaGFsZiBvZiB0aGUgc2VtZXN0ZXIsIGFuZCBjb21wYXJlIHRoZSBtZWFuIG9mIHRoYXQgc2FtcGxlIHRvIHRoZSBtZWFuIG9mIHRoZSBkYXRhc2V0IGJlaW5nIHNhbXBsZWQgZnJvbToNCg0KYGBge3J9DQptZWFuKE1hY1VzZXJzMXN0SGFsZikNCnNldC5zZWVkKDE0KQ0KTWFjVXNlcnNTYW1wbGUuMTwtIHNhbXBsZShNYWNVc2VyczFzdEhhbGYsIDEwKQ0KTWFjVXNlcnNTYW1wbGUuMQ0KbWVhbihNYWNVc2Vyc1NhbXBsZS4xKQ0KYGBgDQpBIHJhbmRvbSBzYW1wbGUgb2YgdGVuIHZhbHVlcyBpcyBub3QgYWx3YXlzIHJlcHJlc2VudGF0aXZlIG9mIHRoZSBsYXJnZXIgc2V0IGZyb20gd2hpY2ggaXQgaXMgZHJhd24uICBJbiB0aGlzIGNhc2UsIHdlIGhhdmUgcmFuZG9tbHkgc2FtcGxlZCB0ZW4gdmFsdWVzIGZyb20gdGhlIDEwMCB2YWx1ZXMgdGhhdCByZXByZXNlbnQgdGhlIG51bWJlciBvZiBpbnN0YW5jZXMgb2YgIltSRURBQ1RFRF0sIGl0IHdvbid0IHdvcmshIiBvZiAxMDAgTWFjIFVzZXJzIG92ZXIgdGhlIGZpcnN0IGhhbGYgb2Ygc2VtZXN0ZXIuICBXZSBrbm93IHRoYXQgdGhlIGF2ZXJhZ2UgbnVtYmVyIG9mIGluc3RhbmNlcyBmb3IgdGhpcyBncm91cCBpcyA1MDAuMzMsIGJ1dCB0aGUgYXZlcmFnZSBvZiBvdXIgdGVuIHNhbXBsZWQgdmFsdWVzIGlzIHJvdWdobHkgNDg2LiAgV2UgY2FuIGltcHJvdmUgdGhlIGFjY3VyYWN5IG9mIHRoZSBtZWFuIGVzdGltYXRlIG9idGFpbmVkIHRocm91Z2ggc2FtcGxpbmcgYnkgdXNpbmcgYm9vdHN0cmFwcGluZy4NCg0KYGBge3J9DQpzZXQuc2VlZCg5OTMpDQpUZW5TYW1wbGVCb290c3RyYXBOUjwtcmVwbGljYXRlKDEwLCBzYW1wbGUoTWFjVXNlcnMxc3RIYWxmLCAxMCwgcmVwbGFjZSA9IEZBTFNFKSkNClRlblNhbXBsZUJvb3RzdHJhcE5SDQpNZWFuVGVuU2FtcGxlQm9vdHN0cmFwTlI8LXJlcGxpY2F0ZSgxMCwgbWVhbihzYW1wbGUoTWFjVXNlcnMxc3RIYWxmLCAxMCwgcmVwbGFjZSA9IEZBTFNFKSkpDQpNZWFuVGVuU2FtcGxlQm9vdHN0cmFwTlINCm1lYW4oTWVhblRlblNhbXBsZUJvb3RzdHJhcE5SKQ0KYGBgDQpUYWtpbmcgMTAgYm9vdHJhcHBlZCBzYW1wbGVzIG9mIHRlbiB2YWx1ZXMsIHdpdGhvdXQgcmVwbGFjZW1lbnQgZWFjaCB5aWVsZHMgYW4gYXZlcmFnZSBvZiByb3VnaGx5IDQ5NiAtIGFuIGltcHJvdmVtZW50LiAgTGV0J3MgdHJ5IHRoZSBzYW1lIHRoaW5nIGZvciBzYW1wbGluZyB3aXRoIHJlcGxhY2VtZW50Og0KYGBge3J9DQpzZXQuc2VlZCg4MikNClRlblNhbXBsZUJvb3N0cmFwV1I8LXJlcGxpY2F0ZSgxMCwgc2FtcGxlKE1hY1VzZXJzMXN0SGFsZiwxMCxyZXBsYWNlID0gVFJVRSkpDQpUZW5TYW1wbGVCb29zdHJhcFdSDQpNZWFuVGVuU2FtcGxlQm9vdHN0cmFwV1I8LXJlcGxpY2F0ZSgxMCwgbWVhbihzYW1wbGUoTWFjVXNlcnMxc3RIYWxmLDEwLHJlcGxhY2UgPSBUUlVFKSkpDQpNZWFuVGVuU2FtcGxlQm9vdHN0cmFwV1INCm1lYW4oTWVhblRlblNhbXBsZUJvb3RzdHJhcFdSKQ0KYGBgDQpIZXJlLCBvdXIgbWVhbiBpcyByb3VnaGx5IDQ5OS4gIEFnYWluLCB0aGlzIGlzIGFuIGltcHJvdmVtZW50IG92ZXIgdGFraW5nIG9ubHkgb25lIHNhbXBsZS4gSG93ZXZlciwgd2UgY2FuIGd1YXJhbnRlZSBldmVyIG1vcmUgYWNjdXJhdGUgZXN0aW1hdGVzIG9mIHRoZSBtZWFuIGJ5IHRha2luZyBldmVuIG1vcmUgYm9vdHN0cmFwcGVkIHNhbXBsZXMsIGFzIHNlZW4gaW4gdGhlIGZvbGxvd2luZyBleGFtcGxlLiANCg0KTm93LCBsZXQncyB0cnkgZXN0aW1hdGluZyB0aGUgbWVhbiBiYXNlZCBvbiAxMDAgYm9vdHN0cmFwcGVkIHNhbXBsZXMgb2YgdGVuIG9ic2VydmF0aW9ucyBlYWNoLCBmaXJzdCB3aXRob3V0IGFuZCB0aGVuIHdpdGggcmVwbGFjZW1lbnQ6DQpgYGB7cn0NCnNldC5zZWVkKDkwKQ0KSHVuZHJlZFNhbXBsZUJvb3RzdHJhcE5SPC1yZXBsaWNhdGUoMTAwLCBzYW1wbGUoTWFjVXNlcnMxc3RIYWxmLCAxMCwgcmVwbGFjZSA9IEZBTFNFKSkNCk1lYW5IdW5kcmVkU2FtcGxlQm9vdHNyYXBOUjwtcmVwbGljYXRlKDEwMCwgbWVhbihzYW1wbGUoTWFjVXNlcnMxc3RIYWxmLCAxMCwgcmVwbGFjZSA9IEZBTFNFKSkpDQpNZWFuSHVuZHJlZFNhbXBsZUJvb3RzcmFwTlINCm1lYW4oTWVhbkh1bmRyZWRTYW1wbGVCb290c3JhcE5SKQ0KYGBgDQoNCmBgYHtyfQ0Kc2V0LnNlZWQoOTIpDQpIdW5kcmVkU2FtcGxlQm9vdHN0cmFwV1I8LXJlcGxpY2F0ZSgxMDAsIHNhbXBsZShNYWNVc2VyczFzdEhhbGYsIDEwLCByZXBsYWNlID0gVFJVRSkpDQpNZWFuSHVuZHJlZFNhbXBsZUJvb3RzdHJhcFdSPC1yZXBsaWNhdGUoMTAwLCBtZWFuKHNhbXBsZShNYWNVc2VyczFzdEhhbGYsIDEwLCByZXBsYWNlID0gVFJVRSkpKQ0KTWVhbkh1bmRyZWRTYW1wbGVCb290c3RyYXBXUg0KbWVhbihNZWFuSHVuZHJlZFNhbXBsZUJvb3RzdHJhcFdSKQ0KYGBgDQpUYWtpbmcgMTAwIGJvb3RzdHJhcHBlZCBzYW1wbGVzIGZ1cnRoZXIgaW1wcm92ZXMgb3VyIGVzdGltYXRlIG9mIHRoZSBtZWFuIG9mIHRoZSBncm91cCB3ZSBhcmUgc2FtcGxpbmcgZnJvbS4gIE5vdywgb3VyIG1lYW4gaXMgdmVyeSBjbG9zZSB0byA1MDAsIHRoZSB0cnVlIHZhbHVlLiAgVGhlIGxhcmdlciB0aGUgc2l6ZSBvZiB0aGUgcG9wdWxhdGlvbiBiZWluZyBzYW1wbGVkIGZyb20sIHRoZSBsZXNzIG9mIGEgZGlmZmVyZW5jZSB0aGVyZSBpcyBiZXR3ZWVuIHNhbXBsaW5nIHdpdGggYW5kIHdpdGhvdXQgcmVwbGFjZW1lbnQuIEluIG91ciBjYXNlLCBhcyB0aGVyZSB3ZXJlIDEwMCB2YWx1ZXMgaW4gdGhlIGdyb3VwIGJlaW5nIHNhbXBsZWQgZnJvbSBhbmQgd2Ugc2FtcGxlZCB0ZW4gdmFsdWVzIGF0IGEgdGltZSwgaXQgZGlkbid0IG1ha2UgYSBsYXJnZSBkaWZmZXJlbmNlIHdoZXRoZXIgd2Ugc2FtcGxlZCB3aXRoIG9yIHdpdGhvdXQgcmVwbGFjZW1lbnQuDQoNClNlZWluZyBhcyB0aGUgcm5vcm0gZnVuY3Rpb24gZ2VuZXJhdGVzIGEgc2FtcGxlIG9mIGEgc3BlY2lmaWVkIG51bWJlciBvZiB2YWx1ZXMgZnJvbSBhIGh5cG90aGV0aWNhbCBub3JtYWwgZGlzdHJpYnV0aW9uIHdpdGggYSB1c2VyLXNwZWNpZmllZCBtZWFuIGFuZCBzdGFuZGFyZCBkZXZpYXRpb24sIGJvb3RzdHJhcHBpbmcgc2ltdWxhdGlvbnMgYXJlIG9mdGVuIHBlcmZvcm1lZCB1c2luZyB0aGUgcm5vcm0gZnVuY3Rpb24gYXMgd2VsbC4NCg0KICAgICAgIA0KIyBBcHBsaWNhdGlvbnMgdG8gR2VuZXJhbCBTdGF0aXN0aWNhbCBQcm9ibGVtcw0KDQojIyBBLiAgRGVtb25zdHJhdGluZyB0aGUgTGF3IG9mIExhcmdlIE51bWJlcnMNCg0KQWNrbm93bGVkZ21lbnQ6IFRoaXMgc2VjdGlvbiBkcmV3IGluc3BpcmF0aW9uIGZyb20gYSBkZW1vbnN0cmF0aW9uIG9mIHRoZSBMYXcgb2YgTGFyZ2UgbnVtYmVycyBjcmVhdGVkIGJ5IFBlZHJvIEYuIFF1aW50YW5hLUFzY2VuY2lvIGFuZCBKYW1lcyBBbmdlbG8gYXQgVUNGLCB3aGljaCBjYW4gYmUgZm91bmQgYXQgdGhlIGZvbGxvd2luZyBVUkw6IGh0dHA6Ly9wYXNjZW5jaW8uY29zLnVjZi5lZHUvY2xhc3Nlcy9NZXRob2RzL1IlMjBkZW1vcy9SJTIwRGVtb25zdHJhdGlvbiUyMExhdyUyMExhcmdlJTIwbnVtYmVycyUyMDIwMTIucGRmDQoNClRoZSBMYXcgb2YgTGFyZ2UgTnVtYmVycyBzdGF0ZXMgdGhhdCBhcyB0aGUgbnVtYmVyIG9mIG9ic2VydmF0aW9ucyBpbiBhIGdpdmVuIHJhbmRvbSBzYW1wbGUgb2YgdGhlIHBvcHVsYXRpb24gaW5jcmVhc2VzLCB0aGUgbWVhbiBvZiB0aGF0IHNhbXBsZSBhcHByb2FjaGVzIHRoZSB0cnVlIG1lYW4gdmFsdWUgb2YgdGhlIHBvcHVsYXRpb24uDQoNCkEgc2ltcGxlIGNvcm9sbGFyeSBjYW4gYmUgbWFkZSB0byB0aGUgYWN0IG9mIGZsaXBwaW5nIGEgY29pbi4gIFdlIGtub3cgdGhhdCB0aGUgdHJ1ZSByYXRlIG9mIGVhY2ggb3V0Y29tZSwgaGVhZHMgb3IgdGFpbHMsIGlzIDAuNS4gSG93ZXZlciwgaWYgd2UgZmxpcCBhIGNvaW4gZml2ZSB0aW1lcywgd2UgY291bGQgZWFzaWx5IG9idGFpbiBhIHJlc3VsdCBvZiBmb3VyIGhlYWRzIGFuZCAxIHRhaWxzLCB5aWVsZGluZyBhIHJhdGUgb2YgMC44IGZvciBoZWFkcyBhbmQgMC4yIGZvciB0YWlscy4gSW4gdGhpcyBjYXNlLCB0aGUgbWVhbiBlc3RpbWF0ZWQgZnJvbSBhIHNhbXBsZSBvZiBmaXZlIHdvdWxkIGJlIGhpZ2hseSBpbm5hY2N1cmF0ZS4gIEhvd2V2ZXIsIGlmIHdlIGZsaXAgYSBjb2luIG9uZSB0aG91c2FuZCB0aW1lcywgaXQgaXMgdmlydHVhbGx5IGltcG9zc2libGUgdG8gb2J0YWluIHN1Y2ggYW4gaW5hY2N1cmF0ZSBlc3RpbWF0ZSBvZiB0aGUgdHJ1ZSByYXRlIG9mIGVhY2ggb3V0Y29tZS4gIEZsaXBwaW5nIGEgY29pbiBvbmUgdGhvdXNhbmQgdGltZXMgd291bGQgeWllbGQgYSByYXRlIHZlcnkgY2xvc2UgdG8gMC41IGZvciBib3RoIGhlYWRzIGFuZCB0YWlscy4NCg0KTGV0J3MgZGVtb25zdHJhdGUgdGhlIExhdyBvZiBMYXJnZSBOdW1iZXJzIGluIFIuICBUbyBiZWdpbiwgd2UgbmVlZCB0byBlc3RhYmxpc2ggYSBwb3B1bGF0aW9uIHRvIHNhbXBsZSBmcm9tLiAgSSBjaG9zZSB0byB1c2UgdGhlIGRpc3RyaWJ1dGlvbiBvZiBKYXBhbmVzZSBtZW5zJyBoZWlnaHRzIGFzIG15IHBvcHVsYXRpb24sIGFzIEphcGFuIHdhcyB0aGUgbW9zdCBwcm9taW5lbnQgY291bnRyeSBmb3Igd2hpY2ggSSBjb3VsZCBmaW5kIGJvdGggYSBtZWFuIGFuZCBhIHN0YW5kYXJkIGRldmlhdGlvbiBmb3IgaGVpZ2h0IChzZWUgaHR0cDovL3d3dy5tZXh0LmdvLmpwL2JfbWVudS90b3VrZWkvMDAxLzAyMi8yMDA0LzAwMi5wZGYpLiAgQmVsb3csIEkgZW50ZXIgdGhlIG1lYW4gb2YgdGhlIHBvcHVsYXRpb24sIDE3MiBjZW50aW1ldGVycywgYXMgYW4gb2JqZWN0ICJtdSIsIGFuZCB0aGUgc3RhbmRhcmQgZGV2aWF0aW9uLCA1LjQyIGNlbnRpbWV0ZXJzLCBhcyBhbiBvYmplY3QgInNpZ21hIi4gIE11IGlzIHRoZSBHcmVlayBsZXR0ZXIgdXNlZCB0byBkZW5vdGUgcG9wdWxhdGlvbiBtZWFuIGFuZCBzaWdtYSBpcyB0aGUgR3JlZWsgbGV0dGVyIHVzZWQgdG8gZGVub3RlIHBvcHVsYXRpb24gc3RhbmRhcmQgZGV2aWF0aW9uDQoNCmBgYHtyfQ0KbXUgPSAxNzINCnNpZ21hID0gNS40Mg0KYGBgDQoNClRvIGRlbW9uc3RyYXRlIHRoZSBMYXcgb2YgTGFyZ2UgbnVtYmVycywgSSB3aWxsIHNhbXBsZSBmcm9tIHRoaXMgcG9wdWxhdGlvbiA1MCB0aW1lcywgd2l0aCB0aGUgbnVtYmVyIG9mIG9ic2VydmF0aW9ucyBwZXIgc2FtcGxlIGluY3JlYXNpbmcgZnJvbSAxIHRvIDUwIGluIGluY3JlbWVudHMgb2Ygb25lLiAgT25jZSBhbGwgb2YgdGhlIHNhbXBsZXMgaGF2ZSBiZWVuIGNvbGxlY3RlZCwgSSB3aWxsIHBsb3QgdGhlIGF2ZXJhZ2Ugb2YgZWFjaCBzYW1wbGUgdG8gb2JzZXJ2ZSB3aGV0aGVyIHRoZSBkaXN0cmlidXRpb24gb2YgbWVhbnMgY2hhbmdlcyBhcyB0aGUgbnVtYmVyIG9mIHNhbXBsZXMgaW5jcmVhc2VzLiAgDQoNCkZpcnN0LCBJIHdpbGwgY3JlYXRlIGFuIGVtcHR5IHZlY3RvciBvZiBsZW5ndGggNTAsIHdoaWNoIHdpbGwgZXZlbnR1YWxseSBiZSBwb3B1bGF0ZWQgYnkgdGhlIG1lYW5zIG9mIG15IDUwIHNhbXBsZXM6DQpgYGB7cn0NClZlY3Rvcl9vZl9NZWFucyA8LSByZXAoMCw1MCkNCmBgYA0KDQpDaGVja2luZyB0aGUgdmVjdG9yLCB3ZSBjYW4gc2VlIHRoYXQgaXQgaXMgY3VycmVudGx5IHBvcHVsYXRlZCBieSA1MCB6ZXJvZXM6DQpgYGB7cn0NClZlY3Rvcl9vZl9NZWFucw0KYGBgDQoNCk5leHQsIEkgbmVlZCB0byBzdGFydCBwb3B1bGF0aW5nIHRoZSB2ZWN0b3IuICBJIHdpbGwgY29uc3RydWN0IHRoZSB2ZWN0b3Igc3VjaCB0aGF0IHRoZSAxc3QgdmFsdWUgd2lsbCBiZSB0aGUgbWVhbiBvZiBhIHNhbXBsZSBvZiAxIG9ic2VydmF0aW9uLCB0aGUgMm5kIHZhbHVlIHdpbGwgYmUgdGhlIG1lYW4gb2YgYSBzYW1wbGUgb2YgMiBvYnNydmF0aW9ucywgZXRjLiAtIGFsbCB0aGUgd2F5IHRvIHRoZSA1MHRoIHZhbHVlIHdoaWNoIGlzIHRoZSBtZWFuIG9mIGEgc2FtcGxlIG9mIDUwIG9ic2VydmF0aW9ucy4gIFRvIGJlZ2luLCBJIG5lZWQgdG8gdGFrZSBhIHNhbXBsZSBvZiBvbmUsIGZpbmQgdGhlIG1lYW4sIGFuZCBlbnRlciB0aGUgbWVhbiBpbnRvIHRoZSB2ZWN0b3Igb2YgYXZlcmFnZXMuICBJIGNvbWJpbmUgdGhlc2Ugc3RlcHMgaW50byBvbmUgZnVuY3Rpb24gYmVsb3c6DQpgYGB7cn0NClZlY3Rvcl9vZl9NZWFuc1sxXTwtbWVhbihybm9ybSgxLG1lYW49bXUsc2Q9c2lnbWEpKQ0KYGBgDQoNCk5vdywgSSBoYXZlIHRvIHJlcGVhdCB0aGUgcHJvY2VzcyBmb3Igc2FtcGxlcyBvZiAyIHRvIDUwIHZhbHVlcyB0byBwb3B1bGF0ZSB0aGUgMm5kIHRocm91Z2ggdGhlIDUwdGggc2xvdCBpbiB0aGUgdmVjdG9yLiBUbyBkbyB0aGlzLCBJIGNvdWxkIHdyaXRlIDQ5IG1vcmUgZXhwcmVzc2lvbnMgbGlrZSB0aGUgb25lIGFib3ZlLCBidXQgYSBtb3JlIGVsZWdhbnQgc29sdXRpb24gaXMgdG8gdXNlIGEgImZvciBsb29wIHRvIGFjY29tcGxpc2ggdGhlIHNhbWUgdGhpbmcuIEZvciBoZWxwIHdpdGggdW5kZXJzdGFuZGluZyBkaWZmZXJlbnQgdHlwZXMgb2YgbG9vcHMgaW4gUiwgc2VlICoqQ2hhcHRlciAzOiBMb29wcyBhbmQgTG9naWNhbHMqKi4gIEluIHRoZSBsb29wIGJlbG93LCBJIGFtIHNwZWNpZnlpbmcgdGhhdCBmb3IgYSB2YWx1ZSBvZiBpIGZyb20gMiB0byA1MCAodGhlIGxlbmd0aCBvZiB0aGUgdmVjdG9yKSwgdGhlIGl0aCB2YWx1ZSBvZiB0aGUgdmVjdG9yIHNob3VsZCBiZSB0aGUgbWVhbiBvZiBhIHNhbXBsZSBvZiBpIHZhbHVlcywgd2l0aCB0aGUgbWVhbiBhbmQgc2QgZXN0YWJsaXNlZCBwcmV2aW91c2x5Og0KYGBge3J9DQpmb3IgKGkgaW4gMjpsZW5ndGgoVmVjdG9yX29mX01lYW5zKSkgew0KICAgIFZlY3Rvcl9vZl9NZWFuc1tpXSA8LSBtZWFuKHJub3JtKGksIG1lYW49bXUsIHNkPXNpZ21hKSkNCn0NCmBgYA0KDQpDaGVja2luZyB0aGUgdmVjdG9yLCB3ZSBjYW4gc2VlIHRoYXQgaXQgaXMgbm93IHBvcHVsYXRlZCBieSB0aGUgNTAgdmFsdWVzIHdlIGhhdmUgZ2VuZXJhdGVkOg0KYGBge3J9DQpWZWN0b3Jfb2ZfTWVhbnMNCmBgYA0KDQpGaW5hbGx5LCBJIGNyZWF0ZSBhIHNjYXR0ZXJwbG90IG9mIHRoZSBtZWFucywgd2l0aCBTYW1wbGUgU2l6ZSBvbiB0aGUgeCBheGlzIGFuZCBDZW50aW1ldGVycyBvbiB0aGUgWSBheGlzLiBUaGUgYmx1ZSBsaW5lIGluZGljYXRlcyB0aGUgdHJ1ZSBtZWFuIG9mIHRoZSBwb3B1bGF0aW9uLCAxNzIgY2VudGltZXRlcnMuICAgRm9yIGhlbHAgd2l0aCBncmFwaGluZyBhbmQgc2NhdHRlcnBsb3RzLCBzZWUgKipDaGFwdGVyIDEwOiBHR3Bsb3QgMSAoU2NhdHRlcnBsb3RzIGFuZCBEZW5zaXR5KSoqLg0KYGBge3J9DQpwbG90KHNlcSgxLDUwKSwgVmVjdG9yX29mX01lYW5zLCB4bGFiID0gIlNhbXBsZSBTaXplIiwgeWxhYiA9ICJDZW50aW1ldGVycyIpDQphYmxpbmUoaD1tdSwgY29sPSJibHVlIikNCmBgYA0KDQpGcm9tIHRoZSBzY2F0dGVycGxvdCwgaXQgaXMgY2xlYXIgdGhhdCB0aGUgZGlzdHJpYnV0aW9uIG9mIHBvaW50cyAoc2FtcGxlIG1lYW5zKSBjbHVzdGVycyBjbG9zZXIgYW5kIGNsb3NlciB0byB0aGUgdHJ1ZSBwb3B1bGF0aW9uIG1lYW4gYXMgc2FtcGxlIHNpemUgaW5jcmVhc2VzLCBkZW1vbnN0cmF0aW5nIHRoZSBMYXcgb2YgTGFyZ2UgTnVtYmVycy4gIEluY3JlYXNpbmcgc2FtcGxlIHNpemUgZnVydGhlciB3b3VsZCBjb250aW51ZSB0aGlzIHRyZW5kLiAgDQoNCiMjIEIuIFNpbXVsYXRpbmcgYSBQb2lzc29uIERpc3RyaWJ1dGlvbg0KDQpBIFBvaXNzb24gZGlzdHJpYnV0aW9uIGlzIGEgZGlzY3JldGUgcHJvYmFiaWxpdHkgZGlzdHJpYnV0aW9uIHRoYXQgZXhwcmVzc2VzIHRoZSBwcm9iYWJpbGl0eSBvZiBhIGdpdmVuIG51bWJlciBvZiBldmVudHMgb2NjdXJyaW5nIGluIGEgZ2l2ZW4gaW50ZXJ2YWwgb2YgdGltZSwgd2l0aCB0aGUgcmVxdWlyZW1lbnQgdGhhdCB0aGVzZSBldmVudHMgb2NjdXIgd2l0aCBhIGtub3duIGF2ZXJhZ2UgcmF0ZSBhbmQgdGhhdCB0aGUgcHJvYmFiaWxpdHkgb2YgZWFjaCBldmVudCBpcyBpbmRlcGVuZGVudCBvZiB0aGUgcHJvYmFiaWxpdHkgb2YgdGhlIHByZXZpb3VzIGV2ZW50cy4gIEluIGEgUG9pc3NvbiBkaXN0cmlidXRpb24gdGhlIHZhcmlhYmxlIGxhbWJkYSwgc29tZXRpbWVzIGNhbGxlZCB0aGUgcmF0ZSBwYXJhbWV0ZXIsIHJlcHJlc2VudHMgdGhlIGF2ZXJhZ2UgbnVtYmVyIG9mIGV2ZW50cyBpbiBhIGdpdmVuIHRpbWUgaW50ZXJ2YWwuDQoNClRoZSBycG9pcygpIGZ1bmN0aW9uIGNhbiBiZSB1c2VkIHRvIHNpbXVsYXRlIGEgUG9pc3NvbiBkaXN0cmlidXRpb24gaW4gUiB1c2luZyB0aGUgZm9sbG93aW5nIHN5bnRheDoNCg0KICAgIGBgYHJwb2lzKG4sIGxhbWJkYSlgYGANCg0KV2hlcmUgbiBpcyB0aGUgbnVtYmVyIG9mIHJhbmRvbSBQb2lzc29uIG51bWJlcnMgdG8gZ2VuZXJhdGUgYW5kIGxhbWJkYSBpcyB0aGUgYWZvcmVtZW50aW9uZWQgcmF0ZSBwYXJhbWV0ZXIgb2YgdGhlIGRpc3RyaWJ1dGlvbi4NCg0KTGV0J3Mgc2ltdWxhdGUgYSBQb2lzc29uIGRpc3RyaWJ1dGlvbiBmb3IgYSByZWFsIHdvcmxkIHNjZW5hcmlvLiBXZSBjYW4gdXNlIGEgUG9pc3NvbiBkaXN0cmlidXRpb24gdG8gc2ltdWxhdGUgdGhlIG51bWJlciBvZiBlbWFpbHMgdGhhdCBhIHR5cGljYWwgVUlDIHBzeWNob2xvZ3kgZ3JhZHVhdGUgc3R1ZGVudCByZWNlaXZlcyBlYWNoIGRheSAoaW5jbHVkaW5nIFVJQyBNQVNTTUFJTCkgb3ZlciBhIDEwMCBkYXkgcGVyaW9kLiBCZWNhdXNlIHRoZSBwcm9iYWJpbGl0eSBvZiBlYWNoIGVtYWlsIHJlY2VpdmVkIGlzIHVzdWFsbHkgaW5kZXBlbmRlbnQgb2YgcHJldmlvdXMgZW1haWxzIHJlY2VpdmVkLCBhIFBvaXNzb24gZGlzdHJpYnV0aW9uIGlzIGEgZ29vZCBjaG9pY2UgdG8gbW9kZWwgdGhpcyBkYXRhLiBMZXQncyBzaW11bGF0ZSBkYXRhIGZvciBhIHR5cGljYWwgc3R1ZGVudCBmb3IgMTAwIGRheXMsIGFzc3VtaW5nIGFuIGF2ZXJhZ2UgcmF0ZSBvZiAzMCBlbWFpbHMgcGVyIGRheSAobGFtYmRhPTMwKToNCg0KYGBge3J9DQpQb2lzc29uRW1haWxzPC1ycG9pcygxMDAsIDMwKQ0KYGBgDQoNClRvIHZpc3VhbGl6ZSB0aGVzZSBkYXRhLCBsZXQncyBtYWtlIGEgc2NhdHRlcnBsb3Q6DQoNCmBgYHtyfQ0KcGxvdChQb2lzc29uRW1haWxzLHhsYWI9IkRheSBOdW1iZXIiLCB5bGFiID0gIk51bWJlciBvZiBFbWFpbHMgUmVjZWl2ZWQiKQ0KYWJsaW5lKGg9MzAsIGNvbD0iYmx1ZSIpDQpgYGANCg0KVGhlIHBsb3QgYWJvdmUgc2hvd3MgdGhlIHNpbXVsYXRlZCBkYXRhIHJlcHJlc2VudGluZyB0aGUgbnVtYmVyIG9mIGVtYWlscyByZWNlaXZlZCBlYWNoIGRheSBvdmVyIHRoZSAxMDAgZGF5IHBlcmlvZC4gIFRoZSBibHVlIGxpbmUgcmVwcmVzZW50cyB0aGUgYXZlcmFnZSByYXRlIG9mIDMwIHdoaWNoIEkgc2V0IGZvciB0aGlzIGRpc3RyaWJ1dGlvbi4gIEFzIGV4cGVjdGVkLCBkYXRhIHBvaW50cyBmb3Igc3BlY2lmaWMgZGF5cyBmYWxsIGJvdGggYWJvdmUgYW5kIGJlbG93IHRoaXMgc3BlY2lmaWVkIGF2ZXJhZ2UgZGFpbHkgcmF0ZS4NCg0KPHNjcmlwdD4NCiAgKGZ1bmN0aW9uKGkscyxvLGcscixhLG0pe2lbJ0dvb2dsZUFuYWx5dGljc09iamVjdCddPXI7aVtyXT1pW3JdfHxmdW5jdGlvbigpew0KICAoaVtyXS5xPWlbcl0ucXx8W10pLnB1c2goYXJndW1lbnRzKX0saVtyXS5sPTEqbmV3IERhdGUoKTthPXMuY3JlYXRlRWxlbWVudChvKSwNCiAgbT1zLmdldEVsZW1lbnRzQnlUYWdOYW1lKG8pWzBdO2EuYXN5bmM9MTthLnNyYz1nO20ucGFyZW50Tm9kZS5pbnNlcnRCZWZvcmUoYSxtKQ0KICB9KSh3aW5kb3csZG9jdW1lbnQsJ3NjcmlwdCcsJ2h0dHBzOi8vd3d3Lmdvb2dsZS1hbmFseXRpY3MuY29tL2FuYWx5dGljcy5qcycsJ2dhJyk7DQoNCiAgZ2EoJ2NyZWF0ZScsICdVQS05ODg3ODc5My0xJywgJ2F1dG8nKTsNCiAgZ2EoJ3NlbmQnLCAncGFnZXZpZXcnKTsNCg0KPC9zY3JpcHQ+DQo=