2 Introduction to R: assignment, vectors, functions, strings, loops

Note: This lecture content was originally created by voluntary contributions to Data Carpentry and has been modified to align with the aims of EEB313. Data Carpentry is an organization focused on data literacy, with the objective of teaching skills to researchers to enable them to retrieve, view, manipulate, analyze, and store their and other’s data in an open and reproducible way.

The above paragraph is made explicit since it is one of the core features of working with an open language like R. Many smart people willingly and actively share their material publicly, so that others can modify and build off of the material themselves.

By being open, we can “stand on the shoulders of giants” and continue to contribute for others to then stand on our shoulders. Not only does this help get work done, but it also adds to a feeling of community. In fact, there is a common saying in the open source world:

I came for the language and stayed for the community.

This saying captures the spirit, generosity, and fun involved in being a part of these open source projects.

2.1 Lesson Preamble

2.1.1 Learning Objectives

Define the following terms as they relate to R: call, function, arguments, options.

Use comments within code blocks.

Do simple arithmetic operations in R using values and objects.

Call functions and use arguments to change their default options.

Define our own functions

Inspect the content of vectors and manipulate their content.

Create for-loops

Describe what a data frame is.

Load external data from a .csv file into a data frame in R.

2.1.2 Lecture outline

Setting up your R Notebook (10 min)

Creating objects/variables in R (10 min)

Using and writing functions (15 min)

Vectors and data types (10 min)

Subsetting vectors (15 min)

Missing data (10 min)

Loops and vectorization (10 min)

Data set background (10 min)

What are data frames? (10 min)

2.2 Setting up the R Notebook

Let’s remove the template RStudio gives us, and add a title of our own.

---
title: Introduction to R
---

This header block is called the YAML header. This is where we specify whether we want to convert this file to a HTML or PDF file. This will be discussed in more detail in another class. For now, we just care about including the lecture title here. If you are interested in playing with other YAML options, check out this guide.

Under this sentence, we will insert our first code chunk. Remember that you insert a code chunk by either clicking the “Insert” button or pressing Ctrl/Cmd + Alt + i simultaneously. To run a code chunk, you press the green arrow, or Ctrl/Cmd + Shift + Enter.

2.3 Creating objects in R

As we saw in our first class, you can get output from R simply by typing math in the console:

3 + 5

[1] 8

12 / 7

[1] 1.714286

However, to do useful and interesting things, we need to assign values to objects.

x <- 3
x + 5

[1] 8

You can name an object in R almost anything you want:

joel <- 3
joel + 5

[1] 8

2.3.0.1 Challenge

So far, we have created two variables, joel and x. What is the sum of these variables?

2.3.1 Some tips on naming objects

Objects can be given any name: x, current_temperature, thing, or subject_id.
You want your object names to be explicit and not too long.
Object names cannot start with a number: x2 is valid, but 2x is not.
R is also case sensitive: joel is different from Joel.
Avoid using the names of existing functions (e.g. mean, df). You can check whether the name is already in use by using tab completion
Generally good to use underscores (_) to separate words in variable and function names

It is also recommended to use nouns for variable names, and verbs for function names. It’s important to be consistent in the styling of your code (where you put spaces, how you name variables, etc.). Using a consistent coding style¹ makes your code clearer to read for your future self and your collaborators. RStudio will format code for you if you highlight a section of code and press Ctrl/Cmd + Shift + a.

2.3.2 Preforming calculations

When assigning a value to an object, R does not print anything. You can force R to print the value by using parentheses or by typing the object name:

weight_kg <- 55    # doesn't print anything
(weight_kg <- 55)  # but putting parentheses around the call prints the value of `weight_kg`

[1] 55

weight_kg          # and so does typing the name of the object

[1] 55

The variable weight_kg is stored in the computer’s memory where R can access it, and we can start doing arithmetic with it efficiently. For instance, we may want to convert this weight into pounds (weight in pounds is 2.2 times the weight in kg):

2.2 * weight_kg

[1] 121

We can also change a variable’s value by assigning it a new one:

weight_kg <- 57.5
2.2 * weight_kg

[1] 126.5

This means that assigning a value to one variable does not change the values of other variables. For example, let’s store the animal’s weight in pounds in a new variable, weight_lb:

weight_lb <- 2.2 * weight_kg # Actually, 1 kg = 2.204623 lbs

and then change weight_kg to 100.

weight_kg <- 100

2.3.2.1 Challenge

What do you think is the current content of the object weight_lb? 126.5 or 220?

weight_lb

2.3.2.2 Challenge

What are the values after each statement in the following?

mass <- 47.5
age  <- 122
mass <- mass * 2.0      # mass?
age  <- age - 20        # age?
mass_index <- mass/age  # mass_index?

2.4 Functions and their arguments

2.4.1 Understanding functions

Functions can be thought of as recipes. You give a few ingredients as input to a function, and it will generate an output based on these ingredients. Just as with baking, both the ingredients and the actual recipe will influence what comes out of the recipe in the end: will it be a cake or a loaf of bread? In R, the inputs to a function are not called ingredients, but rather arguments, and the output is called the return value of the function. A function does not technically have to return a value, but often does so. Functions are used to automate more complicated sets of commands and many of them are already predefined in R. A typical example would be the function sqrt(). The input (the argument) must be a number, and the return value (in fact, the output) is the square root of that number. Executing a function (‘running it’) is called calling the function. An example of a function call is:

sqrt(9)

[1] 3

Which is the same as assigning the value to a variable and then passing that variable to the function:

a <- 9
b <- sqrt(a)
b

[1] 3

Here, the value of a is given to the sqrt() function, the sqrt() function calculates the square root, and returns the value which is then assigned to variable b. This function is very simple, because it takes just one argument.

The return ‘value’ of a function need not be numerical (like that of sqrt()), and it also does not need to be a single item: it can be a set of things, or even a dataset, as we will see later on.

Arguments can be anything, not only numbers or filenames, but also other objects. Exactly what each argument means differs per function, and must be looked up in the documentation (see below). Some functions take arguments which may either be specified by the user, or, if left out, take on a default value: these are called options. Options are typically used to alter the way the function operates, such as whether it ignores ‘bad values’, or what symbol to use in a plot. However, if you want something specific, you can specify a value of your choice which will be used instead of the default.

2.4.2 Tab-completion

To access help about sqrt, we are first going to learn about tab-completion. Type s and press Tab.

s<tab>q

You can see that R gives you suggestions of what functions and variables are available that start with the letter s, and thanks to RStudio they are formatted in this nice list. There are many suggestions here, so let’s be a bit more specific and append a q, to find what we want. If we press enter or tab again, R will insert the selected option.

You can see that R inserts a pair of parentheses together with the name of the function. This is how the function syntax looks for R and many other programming languages, and it means that within these parentheses, we will specify all the arguments (the ingredients) that we want to pass to this function.

If we press tab again, R will helpfully display all the available parameters for this function that we can pass an argument to. The word parameter is used to describe the name that the argument can be passed to. More on that later.

sqrt(<tab>

There are many things in this list, but only one of them is marked in purple. Purple here means that this list item is a parameter we can use for the function, while yellow means that it is a variable that we defined earlier.²

2.4.3 Help with defined functions

To read the full help about sqrt, we can use the question mark, or type it directly into the help document browser.

?sqrt

As you can see, sqrt() takes only one argument, x, which needs to be a numerical vector. Don’t worry too much about the fact that it says vector here; we will talk more about that later. Briefly, a numerical vector is one or more numbers. In R, every number is a vector, so you don’t have to do anything special to create a vector. More on vectors later.

Let’s try a function that can take multiple arguments: round().

round(<tab>)
?round

If we try round with a value:

round(3.14159)

[1] 3

Here, we’ve called round() with just one argument, 3.14159, and it has returned the value 3. That’s because the default is to round to the nearest whole number, or integer. If we want more digits we can pass an argument to the digits parameter, to specify how many decimals we want to round to.

round(3.14159, digits = 2)

[1] 3.14

So, above we pass the argument 2, to the parameter digits. Knowing this nomenclature is not essential for doing your own data analysis, but it will be very helpful when you are reading through help documents online and in RStudio.

We can leave out the word digits since we know it comes as the second parameter, after x.

round(3.14159, 2)

[1] 3.14

As you notice, we have been leaving out x from the beginning. If you provide the names for both the arguments, we can switch their order:

round(digits = 2, x = 3.14159)

[1] 3.14

It’s good practice to put the non-optional arguments (like the number you’re rounding) first in your function call, and to specify the names of all optional arguments. If you don’t, someone reading your code might have to look up the definition of a function with unfamiliar arguments to understand what you’re doing.

2.5 Writing functions

In this class, you will be working a lot with functions, especially those that someone else has already written. When you type sum, c(), or mean(), you are using a function that has been made previously and built into R. To remove some of the magic around these functions, we will go through how to make a basic function of our own. Let’s start with a simple example where we add two numbers together:

add_two_numbers <- function(num1, num2) {
    return(num1 + num2)
}
add_two_numbers(4, 5)

[1] 9

As you can see, running this function on two numbers returns their sum. We could also assign to a variable in the function and return the function.

add_two_numbers <- function(num1, num2) {
    my_sum <- num1 + num2
    return(my_sum)
}
add_two_numbers(4, 5)

[1] 9

2.5.0.1 Challenge

Can you write a function that calculates the mean of 3 numbers?

2.6 Vectors and data types

A vector is the most common and basic data type in R, and is pretty much the workhorse of R. A vector is composed by a series of values, which can be either numbers or characters. We can assign a series of values to a vector using the c() function, which stands for “concatenate (combine/connect one after another) values into a vector” For example we can create a vector of animal weights and assign it to a new object weight_g:

weight_g <- c(50, 60, 65, 82) # Concatenate/Combine values into a vector
weight_g

[1] 50 60 65 82

You can also use the built-in command seq, to create a sequence of numbers without typing all of them in manually.

seq(0, 30) # This is the same as just `0:30`

 [1]  0  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
[26] 25 26 27 28 29 30

seq(0, 30, 3) # Every third number

 [1]  0  3  6  9 12 15 18 21 24 27 30

A vector can also contain characters:

animals <- c('mouse', 'rat', 'dog')
animals

[1] "mouse" "rat"   "dog"

The quotes around “mouse”, “rat”, etc. are essential here and can be either single or double quotes. Without the quotes R will assume there are objects called mouse, rat and dog. As these objects don’t exist in R’s memory, there will be an error message.

There are many functions that allow you to inspect the content of a vector. length() tells you how many elements are in a particular vector:

length(weight_g)

[1] 4

length(animals)

[1] 3

An important feature of a vector is that all of the elements are the same type of data. The function class() indicates the class (the type of element) of an object:

class(weight_g)

[1] "numeric"

class(animals)

[1] "character"

The function str() provides an overview of the structure of an object and its elements. It is a useful function when working with large and complex objects:

str(weight_g)

 num [1:4] 50 60 65 82

str(animals)

 chr [1:3] "mouse" "rat" "dog"

You can use the c() function to add other elements to your vector:

weight_g <- c(weight_g, 90) # add to the end of the vector
weight_g <- c(30, weight_g) # add to the beginning of the vector
weight_g

[1] 30 50 60 65 82 90

In the first line, we take the original vector weight_g, add the value 90 to the end of it, and save the result back into weight_g. Then we add the value 30 to the beginning, again saving the result back into weight_g.

We can do this over and over again to grow a vector, or assemble a dataset. As we program, this may be useful to add results that we are collecting or calculating.

An atomic vector is the simplest R data type and it is a linear vector of a single type, e.g. all numbers. Above, we saw 2 of the 6 main atomic vector types that R uses: "character" and "numeric" (or "double"). These are the basic building blocks that all R objects are built from.

Vectors are one of the many data structures that R uses. Other important ones are lists (list), matrices (matrix), data frames (data.frame), factors (factor) and arrays (array). In this class, we will focus on data frames, which is most commonly used one for data analyses.

2.6.0.1 Challenge

We’ve seen that atomic vectors can be of type character, numeric (or double), integer, and logical. But what happens if we try to mix these types in a single vector? Find out by using class to test these examples.

num_char <- c(1, 2, 3, 'a')
num_logical <- c(1, 2, 3, TRUE)
char_logical <- c('a', 'b', 'c', TRUE)
tricky <- c(1, 2, 3, '4')

This happens because vectors can be of only one data type. Instead of throwing an error and saying that you are trying to mix different types in the same vector, R tries to convert (coerce) the content of this vector to find a “common denominator”. A logical can be turn into 1 or 0, and a number can be turned into a string/character representation. It would be difficult to do it the other way around: would 5 be TRUE or FALSE? What number would ‘t’ be?

In R, we call converting objects from one class into another class coercion. These conversions happen according to a hierarchy, whereby some types get preferentially coerced into other types. Can you draw a diagram that represents the hierarchy of how these data types are coerced?

This can be important to watch for in data sets that you import.

2.7 Subsetting vectors

If we want to extract one or several values from a vector, we must provide one or several indices in square brackets. For instance:

animals <- c("mouse", "rat", "dog", "cat")
animals[2]

[1] "rat"

animals[c(3, 2)]

[1] "dog" "rat"

We can also repeat the indices to create an object with more elements than the original one:

more_animals <- animals[c(1, 2, 3, 2, 1, 4)]
more_animals

[1] "mouse" "rat"   "dog"   "rat"   "mouse" "cat"

R indices start at 1. Programming languages like Fortran, MATLAB, Julia, and R start counting at 1, because that’s what human beings typically do. Languages in the C family (including C++, Java, Perl, and Python) count from 0 because that was historically simpler for computers and can allow for more elegant code.

2.7.1 Conditional subsetting

Another common way of subsetting is by using a logical vector. TRUE will select the element with the same index, while FALSE will not:

weight_g <- c(21, 34, 39, 54, 55)
weight_g[c(TRUE, FALSE, TRUE, TRUE, FALSE)]

[1] 21 39 54

Typically, these logical vectors are not typed by hand, but are the output of other functions or logical tests. For instance, if you wanted to select only the values above 50:

weight_g > 50    # will return logicals with TRUE for the indices that meet the condition

[1] FALSE FALSE FALSE  TRUE  TRUE

## so we can use this to select only the values above 50
weight_g[weight_g > 50]

[1] 54 55

We will consider conditions in more detail in the next few lectures.

2.7.2 Strings (character vectors)

Just a small note about character vectors, also called strings. There are built-in packages for subsetting them that we’ll learn about later. They can be particularly relevant for ecological and genomic data because important data can be nested in complicated strings of text (ex: extracting only the observations that occurred in wet habitats from a column of habitat descriptions or only genes with functions related to drought tolerance).

string1 <- "This is a string" # you can include spaces between your quotes
string2 <- c(string1, "so is this") # concatenate with another string
string2[2] # can access the second string via subsetting

[1] "so is this"

# Playing a bit with declaring variables
"You can include 'quotes' in a string"

[1] "You can include 'quotes' in a string"

string3 <- 'You can include "quotes" in a string'
string3

[1] "You can include \"quotes\" in a string"

"You can include \"matching quotes\" if you 'escape' them with a backslash (\\)"

[1] "You can include \"matching quotes\" if you 'escape' them with a backslash (\\)"

2.8 Missing data

As R was designed to analyze datasets, it includes the concept of missing data (which is uncommon in other programming languages). Missing data are represented in vectors as NA.

When doing operations on numbers, most functions will return NA if the data you are working with include missing values. This feature makes it harder to overlook the cases where you are dealing with missing data. You can add the argument na.rm = TRUE to calculate the result while ignoring the missing values.

heights <- c(2, 4, 4, NA, 6)
mean(heights)

[1] NA

max(heights)

[1] NA

mean(heights, na.rm = TRUE)

[1] 4

max(heights, na.rm = TRUE)

[1] 6

## Extract those elements which are not missing values.
heights[!is.na(heights)]

[1] 2 4 4 6

## Returns the object with incomplete cases removed. The returned object is an atomic vector of type `"numeric"` (or `"double"`).
na.omit(heights)

[1] 2 4 4 6
attr(,"na.action")
[1] 4
attr(,"class")
[1] "omit"

## Extract those elements which are complete cases. The returned object is an atomic vector of type `"numeric"` (or `"double"`).
heights[complete.cases(heights)]

[1] 2 4 4 6

Recall that you can use the class() function to find the type of your atomic vector.

2.8.0.1 Challenge

Using this vector of length measurements, create a new vector with the NAs removed.

lengths <- c(10, 24, NA, 18, NA, 20)

Use the function median() to calculate the median of the lengths vector.

2.9 Loops and vectorization

Loops, specifically for-loops, are essential to programming in general. However, in R, you should avoid them as often as possible because there are more efficient ways of doing things that you should use instead. It is still important that you understand the concept of loops and you might also use them in some of your own functions if there is no vectorized way of going about what you want to do.

You can think of a for-loop as: “for each number contained in a list/vector, perform this operation” and the syntax basically says the same thing:

v <- c(2, 4, 6)
for (num in v) {
    print(num)
}

[1] 2
[1] 4
[1] 6

Instead of printing out every number to the console, we could also add numbers cumulatively, to calculate the sum of all the numbers in the vector:

# To increment `w` each time, we must first create the variable,
# which we do by setting `w <- 0`, referred to as initializing.
# This also ensures that `w` is zero at the start of the loop and
# doesn't retain the value from last time we ran this code.
w <- 0
for (num in v) {
    w <- w + num
}
w

[1] 12

If we put what we just did inside a function, we have essentially recreated the sum function in R.

my_sum <- function(input_vector) {
    vector_sum <- 0
    for (num in input_vector){
        vector_sum <- vector_sum + num
    }
    return(vector_sum)
}

my_sum(v)

[1] 12

Although this gives us the same output as the built-in function sum, the built-in function has many more optimizations so it is much faster than our function. In R, it is always faster to try to find a way of doing things without writing a loop yourself. When you are reading about R, you might see suggestions that you should try to vectorize your code to make it faster. What people are referring to, is that you should not write for loops in R and instead use the ready-made functions that are much more efficient in working with vectors and essentially performs operations on entire vector at once instead of one number at a time. Conceptually, loops operate on one element at a time while vectorized code operates on all elements of a vector at once.

2.10 Dataset background

Today, we will be working with real data from a longitudinal study of the species abundance in the Chihuahuan desert ecosystem near Portal, Arizona, USA. This study includes observations of plants, ants, and rodents from 1977 - 2002, and has been used in over 100 publications. More information is available in the abstract of this paper from 2009. There are several datasets available related to this study, and we will be working with datasets that have been preprocessed by the Data Carpentry to facilitate teaching. These are made available online as The Portal Project Teaching Database, both at the Data Carpentry website, and on Figshare. Figshare is a great place to publish data, code, figures, and more openly to make them available for other researchers and to communicate findings that are not part of a longer paper.

2.10.1 Presentation of the survey data

We are studying the species and weight of animals caught in plots in our study area. The dataset is stored as a comma separated value (CSV) file. Each row holds information for a single animal, and the columns represent:

Column	Description
record_id	unique id for the observation
month	month of observation
day	day of observation
year	year of observation
plot_id	ID of a particular plot
species_id	2-letter code
sex	sex of animal (“M”, “F”)
hindfoot_length	length of the hindfoot in mm
weight	weight of the animal in grams
genus	genus of animal
species	species of animal
taxa	e.g. rodent, reptile, bird, rabbit
plot_type	type of plot

To read the data into R, we are going to use a function called read_csv. This function is contained in an R-package called readr. R-packages are a bit like browser extensions; they are not essential, but can provide nifty functionality. We will go through R-packages in general and which ones are good for data analyses. One useful option that read_csv includes, is the ability to read a CSV file directly from a URL, without downloading it in a separate step:

surveys <- readr::read_csv('https://ndownloader.figshare.com/files/2292169')

However, it is often a good idea to download the data first, so you have a copy stored locally on your computer in case you want to do some offline analyses, or the online version of the file changes or the file is taken down. You can either download the data manually or from within R:

download.file("https://ndownloader.figshare.com/files/2292169",
              "data/portal_data.csv") # Saves to current directory with this name

The data is read in by specifying its local path.

surveys <- readr::read_csv('data/portal_data.csv')

Rows: 34786 Columns: 13
── Column specification ────────────────────────────────────────────────────────
Delimiter: ","
chr (6): species_id, sex, genus, species, taxa, plot_type
dbl (7): record_id, month, day, year, plot_id, hindfoot_length, weight

ℹ Use `spec()` to retrieve the full column specification for this data.
ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.

This statement produces some output regarding which data type it found in each column. If we want to check this in more detail, we can print the variable’s value: surveys.

surveys

# A tibble: 34,786 × 13
   record_id month   day  year plot_id species_id sex   hindfoot_length weight
       <dbl> <dbl> <dbl> <dbl>   <dbl> <chr>      <chr>           <dbl>  <dbl>
 1         1     7    16  1977       2 NL         M                  32     NA
 2        72     8    19  1977       2 NL         M                  31     NA
 3       224     9    13  1977       2 NL         <NA>               NA     NA
 4       266    10    16  1977       2 NL         <NA>               NA     NA
 5       349    11    12  1977       2 NL         <NA>               NA     NA
 6       363    11    12  1977       2 NL         <NA>               NA     NA
 7       435    12    10  1977       2 NL         <NA>               NA     NA
 8       506     1     8  1978       2 NL         <NA>               NA     NA
 9       588     2    18  1978       2 NL         M                  NA    218
10       661     3    11  1978       2 NL         <NA>               NA     NA
# ℹ 34,776 more rows
# ℹ 4 more variables: genus <chr>, species <chr>, taxa <chr>, plot_type <chr>

This displays a nice tabular view of the data, which also includes pagination when there are many rows and we can click the arrow to view all the columns. Technically, this object is actually a tibble rather than a data frame, as indicated in the output. The reason for this is that read_csv automatically converts the data into to a tibble when loading it. Since a tibble is just a data frame with some convenient extra functionality, we will use these words interchangeably from now on.

If we just want to glance at how the data frame looks, it is sufficient to display only the top (the first 6 lines) using the function head():

head(surveys)

# A tibble: 6 × 13
  record_id month   day  year plot_id species_id sex   hindfoot_length weight
      <dbl> <dbl> <dbl> <dbl>   <dbl> <chr>      <chr>           <dbl>  <dbl>
1         1     7    16  1977       2 NL         M                  32     NA
2        72     8    19  1977       2 NL         M                  31     NA
3       224     9    13  1977       2 NL         <NA>               NA     NA
4       266    10    16  1977       2 NL         <NA>               NA     NA
5       349    11    12  1977       2 NL         <NA>               NA     NA
6       363    11    12  1977       2 NL         <NA>               NA     NA
# ℹ 4 more variables: genus <chr>, species <chr>, taxa <chr>, plot_type <chr>

2.11 What are data frames?

Data frames are the de facto data structure for most tabular data, and what we use for statistics and plotting. A data frame can be created by hand, but most commonly they are generated by the function read_csv(); in other words, when importing spreadsheets from your hard drive (or the web).

A data frame is a representation of data in the format of a table where the columns are vectors that all have the same length. Because the columns are vectors, they all contain the same type of data as we discussed in last class (e.g., characters, integers, factors). We can see this when inspecting the structure of a data frame with the function str():

str(surveys)

spc_tbl_ [34,786 × 13] (S3: spec_tbl_df/tbl_df/tbl/data.frame)
 $ record_id      : num [1:34786] 1 72 224 266 349 363 435 506 588 661 ...
 $ month          : num [1:34786] 7 8 9 10 11 11 12 1 2 3 ...
 $ day            : num [1:34786] 16 19 13 16 12 12 10 8 18 11 ...
 $ year           : num [1:34786] 1977 1977 1977 1977 1977 ...
 $ plot_id        : num [1:34786] 2 2 2 2 2 2 2 2 2 2 ...
 $ species_id     : chr [1:34786] "NL" "NL" "NL" "NL" ...
 $ sex            : chr [1:34786] "M" "M" NA NA ...
 $ hindfoot_length: num [1:34786] 32 31 NA NA NA NA NA NA NA NA ...
 $ weight         : num [1:34786] NA NA NA NA NA NA NA NA 218 NA ...
 $ genus          : chr [1:34786] "Neotoma" "Neotoma" "Neotoma" "Neotoma" ...
 $ species        : chr [1:34786] "albigula" "albigula" "albigula" "albigula" ...
 $ taxa           : chr [1:34786] "Rodent" "Rodent" "Rodent" "Rodent" ...
 $ plot_type      : chr [1:34786] "Control" "Control" "Control" "Control" ...
 - attr(*, "spec")=
  .. cols(
  ..   record_id = col_double(),
  ..   month = col_double(),
  ..   day = col_double(),
  ..   year = col_double(),
  ..   plot_id = col_double(),
  ..   species_id = col_character(),
  ..   sex = col_character(),
  ..   hindfoot_length = col_double(),
  ..   weight = col_double(),
  ..   genus = col_character(),
  ..   species = col_character(),
  ..   taxa = col_character(),
  ..   plot_type = col_character()
  .. )
 - attr(*, "problems")=<externalptr>

Integer refers to a whole number, such as 1, 2, 3, 4, etc. Numbers with decimals, 1.0, 2.4, 3.333, are referred to as floats. Factors are used to represent categorical data. Factors can be ordered or unordered, and understanding them is necessary for statistical analysis and for plotting. Factors are stored as integers, and have labels (text) associated with these unique integers. While factors look (and often behave) like character vectors, they are actually integers under the hood, and you need to be careful when treating them like strings.

2.11.1 Inspecting `data.frame` objects

We already saw how the functions head() and str() can be useful to check the content and the structure of a data frame. Here is a non-exhaustive list of functions to get a sense of the content/structure of the data. Let’s try them out!

Size:
- dim(surveys) - returns a vector with the number of rows in the first element and the number of columns as the second element (the dimensions of the object)
- nrow(surveys) - returns the number of rows
- ncol(surveys) - returns the number of columns
Content:
- head(surveys) - shows the first 6 rows
- tail(surveys) - shows the last 6 rows
Names:
- names(surveys) - returns the column names (synonym of colnames() for data.frame objects)
- rownames(surveys) - returns the row names
Summary:
- str(surveys) - structure of the object and information about the class, length, and content of each column
- summary(surveys) - summary statistics for each column

Note: most of these functions are “generic”, they can be used on other types of objects besides data.frame.

2.11.1.1 Challenge

Based on the output of str(surveys), can you answer the following questions?

What is the class of the object surveys?
How many rows and how many columns are in this object?
How many species have been recorded during these surveys?

2.11.2 Indexing and subsetting data frames

Our survey data frame has rows and columns (it has 2 dimensions). If we want to extract some specific data from it, we need to specify the “coordinates” we want from it. Row numbers come first, followed by column numbers. When indexing, base R data frames return a different format depending on how we index the data (i.e. either a vector or a data frame), but with enhanced data frames, tibbles, the returned object is almost always a data frame.

surveys[1, 1]   # first element in the first column of the data frame

# A tibble: 1 × 1
  record_id
      <dbl>
1         1

surveys[1, 6]   # first element in the 6th column

# A tibble: 1 × 1
  species_id
  <chr>     
1 NL

surveys[, 1]    # first column in the data frame

# A tibble: 34,786 × 1
   record_id
       <dbl>
 1         1
 2        72
 3       224
 4       266
 5       349
 6       363
 7       435
 8       506
 9       588
10       661
# ℹ 34,776 more rows

surveys[1]      # first column in the data frame

# A tibble: 34,786 × 1
   record_id
       <dbl>
 1         1
 2        72
 3       224
 4       266
 5       349
 6       363
 7       435
 8       506
 9       588
10       661
# ℹ 34,776 more rows

surveys[1:3, 7] # first three elements in the 7th column

# A tibble: 3 × 1
  sex  
  <chr>
1 M    
2 M    
3 <NA>

surveys[3, ]    # the 3rd element for all columns

# A tibble: 1 × 13
  record_id month   day  year plot_id species_id sex   hindfoot_length weight
      <dbl> <dbl> <dbl> <dbl>   <dbl> <chr>      <chr>           <dbl>  <dbl>
1       224     9    13  1977       2 NL         <NA>               NA     NA
# ℹ 4 more variables: genus <chr>, species <chr>, taxa <chr>, plot_type <chr>

surveys[1:6, ]  # equivalent to head(surveys)

# A tibble: 6 × 13
  record_id month   day  year plot_id species_id sex   hindfoot_length weight
      <dbl> <dbl> <dbl> <dbl>   <dbl> <chr>      <chr>           <dbl>  <dbl>
1         1     7    16  1977       2 NL         M                  32     NA
2        72     8    19  1977       2 NL         M                  31     NA
3       224     9    13  1977       2 NL         <NA>               NA     NA
4       266    10    16  1977       2 NL         <NA>               NA     NA
5       349    11    12  1977       2 NL         <NA>               NA     NA
6       363    11    12  1977       2 NL         <NA>               NA     NA
# ℹ 4 more variables: genus <chr>, species <chr>, taxa <chr>, plot_type <chr>

: is a special operator that creates numeric vectors of integers in increasing or decreasing order; test 1:10 and 10:1 for instance. This works similarly to seq, which we looked at earlier in class:

0:10

 [1]  0  1  2  3  4  5  6  7  8  9 10

seq(0, 10)

 [1]  0  1  2  3  4  5  6  7  8  9 10

# We can test if all elements are the same
0:10 == seq(0,10)

 [1] TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE

all(0:10 == seq(0,10))

[1] TRUE

You can also exclude certain parts of a data frame using the “-” sign:

surveys[,-1]    # All columns, except the first

# A tibble: 34,786 × 12
   month   day  year plot_id species_id sex   hindfoot_length weight genus  
   <dbl> <dbl> <dbl>   <dbl> <chr>      <chr>           <dbl>  <dbl> <chr>  
 1     7    16  1977       2 NL         M                  32     NA Neotoma
 2     8    19  1977       2 NL         M                  31     NA Neotoma
 3     9    13  1977       2 NL         <NA>               NA     NA Neotoma
 4    10    16  1977       2 NL         <NA>               NA     NA Neotoma
 5    11    12  1977       2 NL         <NA>               NA     NA Neotoma
 6    11    12  1977       2 NL         <NA>               NA     NA Neotoma
 7    12    10  1977       2 NL         <NA>               NA     NA Neotoma
 8     1     8  1978       2 NL         <NA>               NA     NA Neotoma
 9     2    18  1978       2 NL         M                  NA    218 Neotoma
10     3    11  1978       2 NL         <NA>               NA     NA Neotoma
# ℹ 34,776 more rows
# ℹ 3 more variables: species <chr>, taxa <chr>, plot_type <chr>

surveys[-c(7:34786),] # Equivalent to head(surveys)

# A tibble: 6 × 13
  record_id month   day  year plot_id species_id sex   hindfoot_length weight
      <dbl> <dbl> <dbl> <dbl>   <dbl> <chr>      <chr>           <dbl>  <dbl>
1         1     7    16  1977       2 NL         M                  32     NA
2        72     8    19  1977       2 NL         M                  31     NA
3       224     9    13  1977       2 NL         <NA>               NA     NA
4       266    10    16  1977       2 NL         <NA>               NA     NA
5       349    11    12  1977       2 NL         <NA>               NA     NA
6       363    11    12  1977       2 NL         <NA>               NA     NA
# ℹ 4 more variables: genus <chr>, species <chr>, taxa <chr>, plot_type <chr>

As well as using numeric values to subset a data.frame (or matrix), columns can be called by name, using one of the four following notations:

surveys["species_id"]       # Result is a data.frame

# A tibble: 34,786 × 1
   species_id
   <chr>     
 1 NL        
 2 NL        
 3 NL        
 4 NL        
 5 NL        
 6 NL        
 7 NL        
 8 NL        
 9 NL        
10 NL        
# ℹ 34,776 more rows

surveys[, "species_id"]     # Result is a data.frame

# A tibble: 34,786 × 1
   species_id
   <chr>     
 1 NL        
 2 NL        
 3 NL        
 4 NL        
 5 NL        
 6 NL        
 7 NL        
 8 NL        
 9 NL        
10 NL        
# ℹ 34,776 more rows

For our purposes, these notations are equivalent. RStudio knows about the columns in your data frame, so you can take advantage of the autocompletion feature to get the full and correct column name.

Another syntax that is often used to specify column names is $. In this case, the returned object is actually a vector. We will not go into detail about this, but since it is such common usage, it is good to be aware of this.

# We use `head()` since the output from vectors are not automatically cut off
# and we don't want to clutter the screen with all the `species_id` values
head(surveys$species_id)          # Result is a vector

[1] "NL" "NL" "NL" "NL" "NL" "NL"

2.11.2.1 Challenge

Create a data.frame (surveys_200) containing only the observations from row 200 of the surveys dataset.
Notice how nrow() gave you the number of rows in a data.frame?
- Use that number to pull out just that last row in the data frame.
- Compare that with what you see as the last row using tail() to make sure it’s meeting expectations.
- Pull out that last row using nrow() instead of the row number.
- Create a new data frame object (surveys_last) from that last row.
Use nrow() to extract the row that is in the middle of the data frame. Store the content of this row in an object named surveys_middle.
Combine nrow() with the - notation above to reproduce the behavior of head(surveys) keeping just the first through 6th rows of the surveys dataset.

2.12 Exporting data

As you begin to play with your raw data, you may want to export these new, processed, datasets to share them with your collaborators or for archival. Similar to the read_csv() function used for reading CSV files into R, there is a write_csv() function that generates CSV files from data frames.

Manually create a new folder called “data-processed” in your directory. Alternatively, get R to help you with it.

dir.create("Processed data")

We are going to prepare a cleaned up version of the data without NAs.

# Note that this omits observations with NA in *any* column.
# There is no way to control which columns to use.
surveys_complete_naomit <- na.omit(surveys)

# Compare the dimensions of the original and the cleaned data frame
dim(surveys)

[1] 34786    13

dim(surveys_complete_naomit)

[1] 30676    13

Now that our dataset is ready, we can save it as a CSV file in our Processed data folder.

# To save to current directory
write_csv(surveys_complete_naomit, "surveys_complete_naomit.csv")

# To save to newly created directory
write_csv(surveys_complete_naomit, 
          file.path("~/Processed data", "surveys_complete_naomit.csv"))

Next lecture, we’re going to discuss collaboration with GitHub and go over an intro to the command line.

Refer to the tidy style guide for which style to adhere to.↩︎
There are a few other symbols as well, all of which can be viewed at the end of this post about RStudio code completion.↩︎