# zoo Quick Reference

This vignette gives a brief overview of (some of) the functionality contained in `zoo` including several nifty code snippets when dealing with (daily) financial data. It was originally written by Ajay Shah. For a more complete overview of the package’s functionality and extensibility see Zeileis and Grothendieck (2005, Journal of Statistical Software, doi:10.18637/jss.v014.i06). (contained as `vignette("zoo", package = "zoo")` in the package), the manual pages and the reference card.

Some of the examples below assume the availability of various data files which are all provided in the package. An easy way to make the available is to set the working directory as follows:

    library("zoo")
    setwd(file.path(system.file(package = "zoo"), "doc"))

## Read a series from a text file

To read in data in a text file, `read.table()` and associated functions can be used as usual with `zoo()` being called subsequently. The convenience function `read.zoo` is a simple wrapper to these functions that assumes the index is in the first column of the file and the remaining columns are data.

Data in `demo1.txt`, where each row looks like

       23 Feb 2005|43.72

can be read in via

    Sys.setlocale("LC_TIME", "C")

    ## [1] "C"

    inrusd <- read.zoo("demo1.txt", sep = "|", format="%d %b %Y")

The `format` argument causes the first column to be transformed to an index of class `"Date"`. The `Sys.setlocale` is set to `"C"` here to assure that the English month abbreviations are read correctly. (It is unnecessary if the system is already set to a locale that understands English month abbreviations.)

The data in `demo2.txt` look like

       Daily,24 Feb 2005,2055.30,4337.00

and requires more attention because of the format of the first column.

    tmp <- read.table("demo2.txt", sep = ",")
    z <- zoo(tmp[, 3:4], as.Date(as.character(tmp[, 2]), format="%d %b %Y"))
    colnames(z) <- c("Nifty", "Junior")

## Query dates

To return all dates corresponding to a series `index(z)` or equivalently

    time(z)

    ##  [1] "2005-02-10" "2005-02-11" "2005-02-14" "2005-02-15" "2005-02-17"
    ##  [6] "2005-02-18" "2005-02-21" "2005-02-22" "2005-02-23" "2005-02-24"
    ## [11] "2005-02-25" "2005-02-28" "2005-03-01" "2005-03-02" "2005-03-03"
    ## [16] "2005-03-04" "2005-03-07" "2005-03-08" "2005-03-09" "2005-03-10"

can be used. The first and last date can be obtained by

    start(z)

    ## [1] "2005-02-10"

    end(inrusd)

    ## [1] "2005-03-10"

## Convert back into a plain matrix

To strip off the dates and just return a plain vector/matrix `coredata` can be used

    plain <- coredata(z)
    str(plain)

    ##  num [1:20, 1:2] 2063 2082 2098 2090 2062 ...
    ##  - attr(*, "dimnames")=List of 2
    ##   ..$ : NULL
    ##   ..$ : chr [1:2] "Nifty" "Junior"

## Union and intersection

Unions and intersections of series can be computed by `merge`. The intersection are those days where both series have time points:

    m <- merge(inrusd, z, all = FALSE)

whereas the union uses all dates and fills the gaps where one series has a time point but the other does not with `NA`s (by default):

    m <- merge(inrusd, z)

`cbind(inrusd, z)` is almost equivalent to the `merge` call, but may lead to inferior naming in some situations hence `merge` is preferred

To combine a series with its lag, use

    merge(inrusd, lag(inrusd, -1))

    ##            inrusd lag(inrusd, -1)
    ## 2005-02-10  43.78              NA
    ## 2005-02-11  43.79           43.78
    ## 2005-02-14  43.72           43.79
    ## 2005-02-15  43.76           43.72
    ## 2005-02-16  43.82           43.76
    ## 2005-02-17  43.74           43.82
    ## 2005-02-18  43.84           43.74
    ## 2005-02-21  43.82           43.84
    ## 2005-02-22  43.72           43.82
    ## 2005-02-23  43.72           43.72
    ## 2005-02-24  43.70           43.72
    ## 2005-02-25  43.69           43.70
    ## 2005-02-28  43.64           43.69
    ## 2005-03-01  43.72           43.64
    ## 2005-03-02  43.70           43.72
    ## 2005-03-03  43.65           43.70
    ## 2005-03-04  43.71           43.65
    ## 2005-03-07  43.69           43.71
    ## 2005-03-09  43.67           43.69
    ## 2005-03-10  43.58           43.67

## Visualization

By default, the `plot()` method generates a graph for each series in `m`

    plot(m)

but several series can also be plotted in a single window.

    plot(m[, 2:3], plot.type = "single", col = c("red", "blue"), lwd = 2)

In addition to the plain base graphics plot() there are methods xyplot() for `lattice`, autoplot() for `ggplot2`, and tinyplot(). All of these provide many additional graphics capabilities, especially including facetting for multivariate series. For example, using the `tinyplot` package one can do:

    library("tinyplot")
    tinyplot(z, facet = Series ~ 1, theme = "clean2",
      palette = "Dark 3", lwd = 2, legend = FALSE)

## Select (a few) observations

Selections can be made for a range of dates of interest

    window(z, start = as.Date("2005-02-15"), end = as.Date("2005-02-28"))

    ##              Nifty  Junior
    ## 2005-02-15 2089.95 4367.25
    ## 2005-02-17 2061.90 4320.15
    ## 2005-02-18 2055.55 4318.15
    ## 2005-02-21 2043.20 4262.25
    ## 2005-02-22 2058.40 4326.10
    ## 2005-02-23 2057.10 4346.00
    ## 2005-02-24 2055.30 4337.00
    ## 2005-02-25 2060.90 4305.75
    ## 2005-02-28 2103.25 4388.20

and also just for a single date

    m[as.Date("2005-03-10")]

    ##            inrusd  Nifty  Junior
    ## 2005-03-10  43.58 2167.4 4648.05

## Handle missing data

Various methods for dealing with `NA`s are available, including linear interpolation

    interpolated <- na.approx(m)

`last observation carried forward’,

    m <- na.locf(m)
    m

    ##            inrusd   Nifty  Junior
    ## 2005-02-10  43.78 2063.35 4379.20
    ## 2005-02-11  43.79 2082.05 4382.90
    ## 2005-02-14  43.72 2098.25 4391.15
    ## 2005-02-15  43.76 2089.95 4367.25
    ## 2005-02-16  43.82 2089.95 4367.25
    ## 2005-02-17  43.74 2061.90 4320.15
    ## 2005-02-18  43.84 2055.55 4318.15
    ## 2005-02-21  43.82 2043.20 4262.25
    ## 2005-02-22  43.72 2058.40 4326.10
    ## 2005-02-23  43.72 2057.10 4346.00
    ## 2005-02-24  43.70 2055.30 4337.00
    ## 2005-02-25  43.69 2060.90 4305.75
    ## 2005-02-28  43.64 2103.25 4388.20
    ## 2005-03-01  43.72 2084.40 4382.25
    ## 2005-03-02  43.70 2093.25 4470.00
    ## 2005-03-03  43.65 2128.85 4515.80
    ## 2005-03-04  43.71 2148.15 4549.55
    ## 2005-03-07  43.69 2160.10 4618.05
    ## 2005-03-08  43.69 2168.95 4666.70
    ## 2005-03-09  43.67 2160.80 4623.85
    ## 2005-03-10  43.58 2167.40 4648.05

and others.

## Prices and returns

To compute log-difference returns in %, the following convenience function is defined

    prices2returns <- function(x) 100*diff(log(x))

which can be used to convert all columns (of prices) into returns.

    r <- prices2returns(m)

A 10-day rolling window standard deviations (for all columns) can be computed by

    rollapply(r, 10, sd)

    ##                inrusd     Nifty    Junior
    ## 2005-02-17 0.14599121 0.6993355 0.7878843
    ## 2005-02-18 0.14527421 0.6300543 0.8083622
    ## 2005-02-21 0.14115862 0.8949318 1.0412806
    ## 2005-02-22 0.15166883 0.9345299 1.0256508
    ## 2005-02-23 0.14285470 0.9454103 1.1957959
    ## 2005-02-24 0.13607992 0.9453855 1.1210963
    ## 2005-02-25 0.11962991 0.9334899 1.1105966
    ## 2005-02-28 0.11963193 0.8585071 0.9388661
    ## 2005-03-01 0.09716262 0.8569891 0.9131822
    ## 2005-03-02 0.09787943 0.8860388 1.0566389
    ## 2005-03-03 0.11568119 0.8659890 1.0176645

To go from a daily series to the series of just the last-traded-day of each month `aggregate` can be used

    prices2returns(aggregate(m, as.yearmon, tail, 1))

    ##              inrusd    Nifty   Junior
    ## Mar 2005 -0.1375831 3.004453 5.752866

Analogously, the series can be aggregated to the last-traded-day of each week employing a convenience function `nextfri` that computes for each `"Date"` the next friday.

    nextfri <- function(x) 7 * ceiling(as.numeric(x-5+4) / 7) + as.Date(5-4)
    prices2returns(aggregate(na.locf(m), nextfri, tail, 1))

    ##                 inrusd      Nifty     Junior
    ## 2005-02-18  0.11411618 -1.2809533 -1.4883536
    ## 2005-02-25 -0.34273997  0.2599329 -0.2875731
    ## 2005-03-04  0.04576659  4.1464226  5.5076988
    ## 2005-03-11 -0.29785794  0.8921286  2.1419450

Here is a similar example of `aggregate` where we define `to4sec` analogously to `nextfri` in order to aggregate the `zoo` object `zsec` every 4 seconds.

    zsec <- structure(1:10, index = structure(c(1234760403.968, 1234760403.969,
    1234760403.969, 1234760405.029, 1234760405.029, 1234760405.03,
    1234760405.03, 1234760405.072, 1234760405.073, 1234760405.073
    ), class = c("POSIXt", "POSIXct"), tzone = ""), class = "zoo")

    to4sec <- function(x) as.POSIXct(4*ceiling(as.numeric(x)/4), origin = "1970-01-01")
    aggregate(zsec, to4sec, tail, 1)

    ## 2009-02-16 05:00:04 2009-02-16 05:00:08
    ##                   3                  10

Here is another example using the same `zsec` zoo object but this time rather than aggregating we truncate times to the second using the last data value for each such second. For large objects this will be much faster than using `aggregate.zoo` .

    # tmp is zsec with time discretized into one second bins
    tmp <- zsec
    st <- start(tmp)
    Epoch <- st - as.numeric(st)
    time(tmp) <- as.integer(time(tmp) + 1e-7) + Epoch

    # find index of last value in each one second interval
    ix <- !duplicated(time(tmp), fromLast = TRUE)

    # merge with grid
    merge(tmp[ix], zoo(, seq(start(tmp), end(tmp), "sec")))

    ## 2009-02-16 05:00:03 2009-02-16 05:00:04 2009-02-16 05:00:05
    ##                   3                  NA                  10

    # Here is a function which generalizes the above:

    intraday.discretise <- function(b, Nsec) {
     st <- start(b)
     time(b) <- Nsec * as.integer(time(b)+1e-7) %/% Nsec + st -
     as.numeric(st)
     ix <- !duplicated(time(b), fromLast = TRUE)
     merge(b[ix], zoo(, seq(start(b), end(b), paste(Nsec, "sec"))))
    }

    intraday.discretise(zsec, 1)

    ## 2009-02-16 05:00:03 2009-02-16 05:00:04 2009-02-16 05:00:05
    ##                   3                  NA                  10

## Query Yahoo! Finance

When connected to the internet, Yahoo! Finance can be easily queried using the `get.hist.quote` function in

    library("tseries")

    if(online) {
      msft <- get.hist.quote(instrument = "MSFT", start = "2004-01-01", end = "2004-12-31")
      msft2 <- get.hist.quote(instrument = "MSFT", start = "2004-01-01", end = "2004-12-31",
        compression = "m", quote = "Close")
      save(msft, msft2, file = "msft2004.rda")
    } else {
      load("msft2004.rda")
    }

From version 0.9-30 on, `get.hist.quote` by default returns `"zoo"` series with a `"Date"` attribute (in previous versions these had to be transformed from `"ts"` `by hand’).

A daily series can be obtained by:

    msft <- get.hist.quote(instrument = "MSFT", start = "2004-01-01", end = "2004-12-31")

A monthly series can be obtained and transformed by

    msft2 <- get.hist.quote(instrument = "MSFT", start = "2004-01-01", end = "2004-12-31",
      compression = "m", quote = "Close")

Here, `"yearmon"` dates might be even more useful:

    time(msft2) <- as.yearmon(time(msft2))

The same series can equivalently be computed from the daily series via

    msft3 <- aggregate(msft[, "Close"], as.yearmon, tail, 1)

The corresponding returns can be computed via

    r <- prices2returns(msft3)

where `r` is still a `"zoo"` series.

## Summaries

Here we create a daily series and then find the series of quarterly means and standard deviations and also for weekly means and standard deviations where we define weeks to end on Tuesay.

We do the above separately for mean and standard deviation, binding the two results together and then show a different approach in which we define a custom `ag` function that can accept multiple function names as a vector argument.

    date1 <- seq(as.Date("2001-01-01"), as.Date("2002-12-1"), by = "day")
    len1 <- length(date1)
    set.seed(1) # to make it reproducible
    data1 <- zoo(rnorm(len1), date1)

    # quarterly summary

    data1q.mean <- aggregate(data1, as.yearqtr, mean)
    data1q.sd <- aggregate(data1, as.yearqtr, sd)
    head(cbind(mean = data1q.mean, sd = data1q.sd), main = "Quarterly")

    ##                 mean        sd
    ## 2001 Q1  0.108503596 0.8861821
    ## 2001 Q2 -0.006836172 0.9800027
    ## 2001 Q3 -0.042260559 0.9954100
    ## 2001 Q4  0.096188411 1.0336234
    ## 2002 Q1 -0.092684201 1.0337850
    ## 2002 Q2  0.049217429 1.0860119

    # weekly summary - week ends on tuesday

    # Given a date find the next Tuesday.
    # Based on formula in Prices and Returns section.
    nexttue <- function(x) 7 * ceiling(as.numeric(x - 2 + 4)/7) + as.Date(2 - 4)

    data1w <- cbind(
           mean = aggregate(data1, nexttue, mean),
           sd = aggregate(data1, nexttue, sd)
    )
    head(data1w)

    ##                   mean        sd
    ## 2001-01-02 -0.22140524 0.5728252
    ## 2001-01-09  0.29574667 0.8686111
    ## 2001-01-16 -0.02281531 1.2305420
    ## 2001-01-23  0.58834959 0.3993624
    ## 2001-01-30 -0.44463015 0.9619139
    ## 2001-02-06 -0.08522653 0.8349544

    ### ALTERNATIVE ###

    # Create function ag like aggregate but takes vector of
    # function names.

    FUNs <- c(mean, sd)
    ag <- function(z, by, FUNs) {
           f <- function(f) aggregate(z, by, f)
           do.call(cbind, sapply(FUNs, f, simplify = FALSE))
    }

    data1q <- ag(data1, as.yearqtr, c("mean", "sd"))
    data1w <- ag(data1, nexttue, c("mean", "sd"))

    head(data1q)

    ##                 mean        sd
    ## 2001 Q1  0.108503596 0.8861821
    ## 2001 Q2 -0.006836172 0.9800027
    ## 2001 Q3 -0.042260559 0.9954100
    ## 2001 Q4  0.096188411 1.0336234
    ## 2002 Q1 -0.092684201 1.0337850
    ## 2002 Q2  0.049217429 1.0860119

    head(data1w)

    ##                   mean        sd
    ## 2001-01-02 -0.22140524 0.5728252
    ## 2001-01-09  0.29574667 0.8686111
    ## 2001-01-16 -0.02281531 1.2305420
    ## 2001-01-23  0.58834959 0.3993624
    ## 2001-01-30 -0.44463015 0.9619139
    ## 2001-02-06 -0.08522653 0.8349544

A convenience function which can determine for a vector of `"Date"` observations whether it is a weekend or not

    is.weekend <- function(x) ((as.numeric(x)-2) %% 7) < 2

The function `is.weekend` introduced above exploits the fact that a `"Date"` is essentially the number of days since 1970-01-01, a Thursday. A more intelligible function which yields identical results could be based on the `"POSIXlt"` class

    is.weekend <- function(x) {
      x <- as.POSIXlt(x)
      x$wday > 5 | x$wday < 1
    }