Arizona State University WGHL481, 4th floor
Apr 18-19, 2015
9:00 am - 4:30 pm
Instructors: Gabriel Devenyi, Rachel Schwartz
Helpers: Tara Furstenau, David Winter, Ryan Middleton
General Information
Software Carpentry's mission is to help scientists and engineers get more research done in less time and with less pain by teaching them basic lab skills for scientific computing. This hands-on workshop will cover basic concepts and tools, including program design, version control, data management, and task automation. Participants will be encouraged to help one another and to apply what they have learned to their own research problems.
Who: The course is aimed at graduate students, post docs, and other researchers, primarily at Arizona State University.
Where: Wrigley Building Room 481, Arizona State University. 800 S Cady Mall, Tempe, AZ 85281. Get directions with OpenStreetMap or Google Maps.
Parking: ASU Parking Information. The Fulton Structure is the closest ($8 / day). However, other lots will be free on weekends. You may be able to park in the church parking lot on the west side of Wrigley Hall. It is $5 per day during the week, but I am not sure if they charge on the weekend. However, Sunday they are likely to be using it for services.
Requirements: Participants must bring a laptop with a few specific software packages installed (listed below). They are also required to abide by Software Carpentry's Code of Conduct.
Host and Sponsors: Hosted by ASU's Wrigley Institute of Sustainability. Sponsored by the iPlant Collaborative and AEGIS.
Contact: Please mail hilgert@email.arizona.edu for more information.
Schedule
Day 1
| 08:30 | Breakfast (provided) |
| 09:00 | Automating tasks with the Unix shell |
| 10:30 | Coffee (provided) |
| 12:00 | Lunch (provided) |
| 13:00 | Building programs with R |
| 14:30 | Iced tea / lemonade (provided) |
| 16:00 | Wrap-up |
Day 2
| 08:30 | Breakfast (provided) |
| 09:00 | Version control with Git |
| 10:30 | Coffee (provided) |
| 12:00 | Lunch (provided) |
| 13:00 | Managing data with SQL |
| 14:30 | Iced tea / lemonade (provided) |
| 16:00 | Wrap-up |
Etherpad: https://etherpad.mozilla.org/nbtmp2fn8p.
We will use this Etherpad for chatting, taking notes, and sharing URLs and bits of code.
This is the saved etherpad.
Syllabus
The Unix Shell
- Files and directories
- History and tab completion
- Pipes and redirection
- Looping over files
- Creating and running shell scripts
- Finding things
- Reference...
Programming in R
- Working with vectors and data frames
- Reading and plotting data
- Creating and using functions
- Loops and conditionals
- Using R from the command line
- Reference...
Version Control with Git
- Creating a repository
- Recording changes to files:
add,commit, ... - Viewing changes:
status,diff, ... - Ignoring files
- Working on the web:
clone,pull,push, ... - Resolving conflicts
- Open licenses
- Where to host work, and why
- Reference...
Managing Data with SQL
- Reading and sorting data
- Filtering with
where - Calculating new values on the fly
- Handling missing values
- Combining values using aggregation
- Combining information from multiple tables using
join - Creating, modifying, and deleting data
- Programming with databases
- Reference...
Setup
This page has instructions on testing that you have the right software installed.
Text Editor
When you're writing code, it's nice to have a text editor that is optimized for writing code, with features like automatic color-coding of key words. The default text editor on Mac OS X and Linux is usually set to Vim, which is not famous for being intuitive. if you accidentally find yourself stuck in it, try typing the escape key, followed by ':q!' (colon, lower-case 'q', exclamation mark), then hitting Return to return to the shell.
Windows
We will be using the open-source software nano, which will be installed as part of the package listed under the sql installation.
The following alternative editors are closed-source, but are very popular alternatives.
Mac OS X
We will be using the open-source software nano, which is pre-installed on Mac. To enable syntax highlighting (I recommend doing this after you feel comfortable on the command line) do the following:
- Download and unzip the syntax files.
- From the command line: go to the directory containing your unzipped files.
- Type: make install
- Type: nano ~/.nanorc
- Paste: include ~/.nano/syntax/ALL.nanorc
- Ctl-X will exit nano; make sure you save the file.
Linux
We will be using the open-source software nano, which is pre-installed on Linux. The following alternative editors are closed-source, but are very popular alternatives.
The Bash Shell
Bash is a commonly-used shell that gives you the power to do simple tasks more quickly.
Download this file and unzip it into a local folder so we can work through an example together.
Windows
Install Git for Windows by download and running the installer. This will provide you with both Git and Bash in the Git Bash program.
Mac OS X
The default shell in all versions of Mac OS X is bash, so no
need to install anything. You access bash from the Terminal
(found in
/Applications/Utilities). You may want to keep
Terminal in your dock for this workshop.
Linux
The default shell is usually bash, but if your
machine is set up differently you can run it by opening a
terminal and typing bash. There is no need to
install anything.
Git
Git is a version control system that lets you track who made changes to what when and has options for easily updating a shared or public version of your code on github.com.
Windows
Git should be installed on your computer as part of your Bash install (described above).
Mac OS X
For OS X 10.8 and higher, install Git for Mac
by downloading and running
the installer.
After installing Git, there will not be anything in your /Applications folder,
as Git is a command line program.
For older versions of OS X (10.5-10.7) use the
most recent available installer for your
OS available
here. Use the Leopard installer for 10.5 and the Snow
Leopard installer for 10.6-10.7.
Linux
If Git is not already available on your machine you can try to
install it via your distro's package manager. For Debian/Ubuntu run
sudo apt-get install git and for Fedora run
sudo yum install git.
R
R is a programming language that is especially powerful for data exploration, visualization, and statistical analysis. To interact with R, we use RStudio.
Windows
Install R by downloading and running this .exe file from CRAN. Also, please install the RStudio IDE.
Mac OS X
Install R by downloading and running this .pkg file from CRAN. Also, please install the RStudio IDE.
Linux
You can download the binary files for your distribution
from CRAN. Or
you can use your package manager (e.g. for Debian/Ubuntu
run sudo apt-get install r-base and for Fedora run
sudo yum install R). Also, please install the
RStudio IDE.
SQLite
SQL is a specialized programming language used with databases. We use a simple database manager called SQLite in our lessons.
Windows
Download the Software Carpentry Windows installer. Double click on the file to run it.
Mac OS X
sqlite3 comes pre-installed on Mac OS X.
Linux
sqlite3 comes pre-installed on Linux.