Links - Data-Intensive Computing for Text Analysis: Fall'11

Links

Suggestions for other links welcome!

UT's Writing Center
Austin Big Data User Group

Hadoop Computing at UT Austin

TACC Hadoop: site that describes how to use Hadoop on the Longhorn computer cluster at TACC.
UTCL Hadoop: site for students in the computational linguistics group to use our small in-house Hadoop cluster

Apache Hadoop: the main site
Hadoop Summits: 2011 & 2010
IEEE Cluster Conference Sept 26-30, 2011 in Austin

Hadoop Tutorial 9/26
Automatic Task Re-organization in MapReduce by Zhenhua Guo, Marlon Pierce, Geoffrey Fox, Mo Zhou (presented during classtime)

Support Tools

Apache Whirr (to launch CDH3 on EC2)
Amazon Elastic Compute Cloud (Amazon EC2)
Cloud9 Library
Cloudera -- see the CDH Users discussion group
Sun Grid Engine v6.2 (job submission system at TACC)

Research Software & Data

Public Data Sets on Amazon
IR Software and Data sets
Ivory: MapReduce IR toolkit by Jimmy Lin
Katta: scalable, failure tolerant, distributed, data storage for real time access to very large data sets, available for Hadoop mapfiles.
ManReduce

Related Courses

Jordan Boyd-Graber, 2011, U. Maryland: Cloud Computing
Jimmy Lin and Nitin Madnani, 2010, U. Maryland: Data-Intensive Information Processing Applications
Miles Osborne, 2010/2011, U. Edinburgh: Extreme Computing
More courses listed on Lin and Dyer book page.

Software Development

Bitbucket (Mercurial code hosting; we'll use in class)
Github (Git code hosting; optional use within your group)
Pivotal Tracker (easy to use project tracker)

Comments