The ROpenSci team has started publishing a new series of interviews with the goal of “demystifying the creative and development processes of R community members”. I had the great pleasure of being interviewed by Kelly O'Briant earlier this year, and the interview was published on Friday.
Thanks for being a great interviewer, Kelly! I'm looking forward to hearing from other R community members as the the rest of the series is published.
ROpenSci blog: .rprofile: David Smith
Thanks to Dr George Esaw, I recently learned that Ross Ihaka, co-creator of R, was featured in a full-page advertisement placed in The Economist by the University of Auckland back in April:
Click on the image for a larger version, and if you can't read the text I've reproduced it here:
He created a language used by millions.
A language called R. And as those millions use it to predict shifts in the stock market, track the behaviour of marine life, or search for a cure for cancer, the effects of this statistical programming language -- developed by Associate Professor Ross Ihaka at his colleagues at the University of Auckland -- reach far beyond millions.
That said, what Ross is focused on now is the future. As datasets grow larger, so too does the need for R to become more powerful.
So where it's being used by an internet giant in California, or a little-known institute near Cape Town, this adaptive, open-source platform known by just a single letter, will allow the new challenges we face to be understood more deeply, and in turn bring about amazing new breakthroughs.
It's great to see that the University of Auckland is drawing on its heritage as the birthplace of R to give a shout-out to one of its founders!
Hadley Wickham, RStudio's Chief Scientist and prolific author of R books and packages, conducted an AMA (Ask Me Anything) session on Reddit this past Monday. The session was tremendously popular, generating more than 500 questions/comments and promoting the AMA to the front page of Reddit.
If you're not familiar with Hadley's work (which would be a surprise if you're an R user), his own introduction in the Reddit AMA post will fill you in:
Broadly, I'm interested in the process of data analysis/science and how to make it easier, faster, and more fun. That's what has led to the development of my most popular packages like ggplot2, dplyr, tidyr, stringr. This year, I've been particularly interested in making it as easy as possible to get data into R. That's lead to my work on the DBI, haven, readr, readxl, and httr packages. Please feel free to ask me anything about the craft of data science.
I'm also broadly interested in the craft of programming, and the design of programming languages. I'm interested in helping people see the beauty at the heart of R and learn to master it as easily as possible. As well as a number of packages like devtools, testthat, and roxygen2, I've written two books along those lines: Advanced R, which teaches R as a programming language, mostly divorced from its usual application as a data analysis tool; and R packages, which teaches software development best practices for R: documentation, unit testing, etc.
Check out the comments at the link below, where you'll find insights from Hadley on the best way to teach R, Big Data in R, the elegance (or otherwise) of the R language, being productive, the best BBQ, and much more.
Priceonomics published on Friday an in-depth profile of Hadley Wickham, author of many of the most popular R packages including ggplot2, dplyr and devtools. In the article, he reveals that his motivation for creating these packages was primarily to provide better ways of accomplishing routine tasks in R, an immensely useful contribution that sadly wasn't recognized in an academic setting. He said:
“There are definitely some academic statisticians who just don’t understand why what I do is statistics, but basically I think they are all wrong . What I do is fundamentally statistics. The fact that data science exists as a field is a colossal failure of statistics. To me, that is what statistics is all about. It is gaining insight from data using modelling and visualization. Data munging and manipulation is hard and statistics has just said that’s not our domain.”
I couldn't agree with the sentiment more, and I too with the field of Statistics had more respect for solving these "mundane" (i.e. non-mathematical), but important problems.
Hadley also says that his work with R has made him "nerd famous" — in the words of the article author, "The kind of famous where people at statistics conferences line up for selfies, ask him for autographs, and are generally in awe of him". I can attest to the truth of that personally: here's a photo I took at the China R User Conference last year, where enthusiastic attendees were lined up at least 5 deep to get his autograph and take pictures:
The whole article is definitely worth a read, and can be found at the link below. You may also like to check out the 2010 profile on Hadley Wickham from this blog.
Priceonomics: Hadley Wickham, the Man Who Revolutionized R
"The R-Files" is an occasional series from Revolution Analytics, where we profile prominent members of the R Community.
|
Name: Paul Teetor Profession: Quantitative developer (freelance) Nationality: American Years Using R: 7 Known for: Author of R Cookbook (O’Reilly Media, 2011) |
An active member of the R community, Paul Teetor is a quantitative developer and statistical consultant based in the Chicago area. He’s been using R for seven years, during which time his contributions to the community have been significant -- particularly in the field of finance. He’s currently a freelance consultant largely focused on time series analysis. Teetor is also the author of the popular R Cookbook, which was published by O’Reilly Media this past March and offers new users over 200 “recipes” for performing more efficient data analysis with R.
He was first drawn to R for the flexibility it offered him in developing trading systems. Citing his own background in software engineering and the need to perform sophisticated statistical analysis in a programmable -- and cost-effective -- environment, Teetor said that R emerged as the perfect fit for him. Since then, he has performed the majority of his financial analyses in R and has also emerged as a leading evangelist for the community. He gradually collected a catalog of tricks and techniques for R, many of which were compiled into the R Cookbook. He's been a participant at conferences such as the Joint Statistical Meetings and the R/Finance Conference where he evangelized the role of R in quantitative finance. Some of those talks and papers are available on his website.
“Prior to R, I did most of my statistical analysis in Excel -- and occasionally SAS,” said Teetor. “However, performing statistical analyses for financial tables in either was extremely tedious and puts you in a specific box. R is a modern, programmable language, so I can make it do what I need it to do in a timely manner. It’s been a pleasure to be able to take what I’ve learned from R and share it with other community members - and to continue learning new tips and tricks from them as well.”
Teetor uses R for the majority of his finance work because, as he puts it, it does things other languages “simply cannot do.” He cited the example of hedge ratio calculations which benefit from the flexibility of R, a topic on which he gave a lightning talk at R/Finance this past summer. He was also quick to credit fellow R user Jeff Ryan (whom we profiled here earlier this year) as an influential member of the R community, citing his finance packages as particularly useful. “I use nearly every finance package he’s written, they’re incredibly helpful and greatly streamline the process of R-based financial analysis.”
When asked about the relationship between financial analysis and the rise of the data science movement, Teetor noted, “People in data science are experiencing what financial analysts have experienced for years: out of the box data analysis is not realistic. You need to incorporate a heavy amount of custom statistics, something that’s not easy to do with a commercial product where you can’t get to the source code. Data scientists need a way to construct custom analyses and R gives them that opportunity. Nothing else on the horizon that can compete with that, in terms of finance or the wider field of data science.”
Looking ahead, Teetor sees a bright future for the continued evolution of R. Since there is no real alternative on the market, he argues, R’s potential for future growth is nearly unlimited. He did, however, cite R’s capacity (or lack thereof) for software engineering as one possible area of improvement. “When R was originally envisioned, it wasn’t thought of as a vehicle for software engineering. Nobody expected people to keep their scripts as opposed to just throwing them away. As it’s grown though, people are building larger, more complex systems with longer lifetimes.” It’s an area that Teetor cites as one of the main struggles with R today, but also one which he cites as a great opportunity on which to innovate.
"The R-Files" is an occasional series from Revolution Analytics, where we profile prominent members of the R Community.
Martyn Plummer is a longtime contributor to the R community and a member of the R core group, which consists of 20 members that help oversee the continued evolution of the project. Plummer also serves on the editorial board of the R Journal, the official journal of the R project. By day, he serves as a Statistician and Epidemiologist at the International Agency for Research on Cancer (IARC), based in Lyon, France.
Plummer, who has been using R since 1995, has developed or contributed to a number of popular packages, including coda for analyzing Markov Chain Monte Carlo output, JAGS, a clone of the popular WinBUGS software Bayesian analysis and Epi, which provides functions for epidemiologists and accompanies an annual course that aims to introduce epidemiologists to R.
He has also incorporated R into his work at IARC, where he works in the Infection and Cancer Epidemiology group. Much of the work of this group is focused on human papillomavirus (HPV), which causes half a million cases of cervical cancer per year worldwide. Plummer and his colleagues use R (including his own Epi package) to analyze epidemiological studies of HPV infection and try to tease out some aspects of HPV natural history that are difficult to understand without statistical modeling, such as whether different HPV types interact with each other. He also relies heavily on R’s graphical capabilities for visualizing data in scientific publications.
Prior to R, Plummer worked primarily with S+ for analyzing data. He had been working in Cambridge, United Kingdom in the Biostatistics Unit at the Medical Research Council when he was offered a position at the IARC in Lyon. He recalls the transition, and how his new position introduced an entirely different computing environment. Soon after moving, he was introduced to the recently-formed R project by his colleague David Clayton.
“From the beginning, I saw enormous potential in R,” says Plummer. “While I was accustomed to S+, it wasn’t long before I completely switched over to R. It was and continues to be unparalleled in its flexibility in terms of data analysis.”
Plummer also points to R’s extensible nature as one of its defining features. As a modern language, R is able to effectively adapt to the changing nature of data analysis in an era of increasingly large, unstructured data sets. “One of the most important features of R is that it’s built around the data; it’s designed for programming with data, so it can take these developments in stride,” he says.
He went on to describe a recent article in the R journal that analyzed 18 months’ worth of text from the R mailing lists and identified relationships between prominent members of the R community based on the topics they discussed. Plummer cites it as an example of R’s ability to keep up with the ever-changing notion of “data.”
“10 years ago, I would have never called such an amalgamation of text a ‘data set,’” he says. “Today, though, we find ourselves in a situation where we can elicit structure from large and complex data sets and glean meaning from it.”
When asked about how he sees the R project evolving in coming years, Plummer speaks of a delicate yet effective balance. “R manages a difficult equilibrium; it’s partly a frontier for innovation in statistical computing, yet it’s also a stable platform for data analysis. It’s unique in this regard and I don’t see it facing serious competition for quite some time.”
He sees the current situation being maintained at least over the next few years, though one challenge for R users is to navigate the increasing number of contributed packages. While there’s incredible innovation being done for a diverse range of functions, Plummer says, there are also opportunities for the community as a whole to pool and share their work.
“One of the most important and oft-overlooked values of the R community is its interdisciplinary nature,” he says. “It’s remarkable to be able to collaborate with so many talented people from a diverse range of fields. We’re all statisticians, but statistics has a terrible tendency to fragment by subject matter. R gives us all a common platform and brings us together to encourage innovation.”
"The R-Files" is an occasional series from Revolution Analytics, where we profile prominent members of the R Community.
|
Name: Jeff Ryan Profession: Owner/Principal at Lemnica; Committee Member at R/Finance Nationality: American Years Using R: 8 Known for: R/Finance Conference, quantmod and xts packages |
Jeffrey Ryan is a Chicago-based quantitative software analyst and avid R user. He is perhaps best known in the R community as one of the primary organizers of the annual R/Finance conference. By day, he’s Principal at Lemnica, a firm that specalizes in developing quantitative software systems for financial market trading. In addition to his efforts on the R/Finance organizing committee (of which Dirk Eddelbuettel, whom we profiled here earlier this year, is also a member), he has also developed a number of popular R packages for financial analysis -- most notably quantmod and xts.
Ryan first started using R in 2002 as an undergraduate at the University of Illinois - Chicago, where he studied economics and finance. He described his frustration with cumbersome proprietary tools and recalls his search for a more extensible programming language with which to work. "At that point, I did most of my work in perl and Excel/VBA," he says. "R was not particularly well known at that point, but I recognized the great potential it had for adding deeper statistical insights into my analyses."
After receiving his Bachelor’s degree from UIC, Ryan went to work for a Chicago-based hedge fund as a floor trader on the options exchange. While he wasn’t actively using R in his work at that point, he continued to experiment in his free time by building trading tools and models in R.
From there, Ryan began using R more frequently and became an increasingly active participant on the R-SIG-Finance mailing list. He recalls being particularly impressed by the work of Diethelm Wuertz, whom he credits as one of his major influences from the R community. Over time, Ryan took the lead on developing an all-in-one R package for quantiative trading models: quantmod.
The quantmod package is designed to assist quantitative traders in developing, testing and deploying trading models and has been adopted by big and small traders around the world. Says Ryan of quantmod, "I come from a quantitative trading background. When I was in school, there was no single package for building and testing a trading model. I saw the chance to do something like that with R, which is where quantmod was born."
"The package has evolved concurrently with R and is one of the more popular packages today – with the website serving as a gateway for new users to R and finance. I’m particularly honored to see the traction quantmod and other tools have gained amongst professional quants in the community, as well as how instrumental it has been in bringing new users into R."
In addition to quantmod, Ryan has developed numerous R packages and has been a contributor to many more. Highlights include xts, or eXtensible Time Series which is used to manage large financial time-series, and IBrokers, which is geared towards real time trading. He is also working on a new collection of packages called “indexing”. According to Ryan, indexing is an abstraction of data.frames that allow fast queries on data that do not fit in memory.
When asked to comment on the significant role he’s played in developing R’s capabilities for quantitative finance, Ryan is quick to credit other members of the R community. In addition to Diethelm Wuertz and Dirk Eddelbuettel, Ryan specifically mentioned Josh Urlich (TTR, LSPM) and Brian Peterson and Peter Carl (PerformanceAnalytics) for the work they’ve done in developing finance-focused packages in R, as well as their key contributions to the success of the R/Finance conferences.
The 2011 R/Finance conference was held this past April 29-30, drawing more than 250 attendees from across the globe. The 2012 conference has already been confirmed for May 17-19 of next year – once again to be held in Ryan’s hometown of Chicago.