Top Data Science Languages

Vincent Granville usefully summarizes a survey of language use.

Top Languages for analytics, data mining, data science:

Very interesting study published by KDNuggets. Here are the highlights:

The most popular languages continue to be R (used by 61% of KDnuggets readers), Python (39%), and SQL (37%). SAS is stable at around 20%. The highest growth was for Pig/Hive/Hadoop-based languages, R, and SQL, while Perl, C/C++, and Unix tools declined. We also find a small affinity between R and Python users. ... Previous KDnuggets polls looked at high-level Analytics and Data mining software, but sometimes a full-power programming language is needed.  ... ' 

Useful yes, but the respondents here are followers of KDNuggets, skewed I believe to researchers rather than practitioners.    By my observation, Excel is still the most used in applications of these ideas, but mentioned nowhere in the article.   I think we need more than Excel to do our job, but we can't ignore its existence.

Statistician vs Data Scientist

Vincent Granville measures the frequency of use of different keywords and finds the use of the word 'statistician' dropping versus 'data analyst'.   Death of the statistician?  I don't think so, but some re-calibration of the meaning of this and related roles like 'data scientist'.   He also explores the use of related terms.

Trusting a Data Scientist

A Challenge: Just because a study contains numbers or big data or statistics does not mean it is correct. So you should not completely trust any scientist for business purpose.  Results from science should be carefully applied and validated to decision and business.  For that you need to know the business well enough to make the call of its correctness or value.

Big Data for the Masses

Can everyone be a data scientist?  You will at very least need the strong support of data scientists. Similar to delivering analytics to everyone.  Its not so much about the technology to extract data from large and volatile sources, but about automating basic analysis and guiding the user to insights.   Not too much different from efforts by Tableau to do the same thing.  But taking this beyond just visualization analytics.  Insights that are useful to their business. A startup at Stanford is trying to deliver this.   Following. " ... 'Through powerful analytic models developed over a decade at Stanford, Ayasdi lets you find the needle in a haystack you didn't know was there, quickly.” Jonathan Ballon, GE’s chief strategy officer, said in a statement. “For GE and our customers with vast amounts of industrial data, Ayasdi's technology will be a powerful tool for predictive analytic models that can drive billions of productivity and efficiency savings."  ... '

On Not Trusting Data Science

Language Lab of the University of Pennsylvania comments on a recent article about the nature of the data scientist.  Essentially making the case that the label of 'scientist' is less valuable than it seems.  Most science in reviewed journals does not provide access to the data used for the work.  So the results are not really reproducible.  So much for the claim of science.  As they state it: " ... Most peer-reviewed scientific papers are based on unpublished (and typically unavailable) data, and under-documented (and often crucially errorful) methods. Journals are reluctant to publish negative results (for the plausible reason that there are lots of ways to screw up an experiment), and equally reluctant to publish failures to replicate positive ones. For these and other reasons, most peer-reviewed scientific papers are wrong, and the more prominent the journal, the less likely published results are to be replicable. ... " 

They give some interesting examples in the language space.  So much for what I thought was a key aspect of science.

NumberSense Reviewed

I have been looking for books that explain the difference of what I call analytics, in my long time consulting practice and the newly hyped arena of Big Data.   I am always also on the look out for books that I can use with client groups to explain how analytics is powerful.    The explanation has to be positioned with little or no complex mathematics.  The examples have to be clear, and beg for their simple reapplication to new business domains.

NumberSense includes these characteristics.   Easy to understand, non technical examples.  In the social/marketing/economic/sports domains.  Clear positioning about how the problems should be staged.  Not much about how the problems are technically solved, but that is for the data technologists.  Not a how to book, but sets up the crucial cautions very clearly.

I particularly liked the analysis of Groupon data, which clearly defines where claims and analyses can be wrong in marketing.   A good marketing analysis example.

Fung appreciates the fact that while having more, or 'big' data is useful, but it is more important to get the data and its analysis right, especially as it relates to the decision problem being addressed.  Numbersense is paying attention to the origin and context of the data involved, and knowing enough about how the analysis will be applied to the real problem.  Misinterpretation is the worst mistake you can make.

In the final chapter Fung describes a  day in his life as a data scientist.  This was painfully reminiscent of some of my own enterprise experiences.  Its often more difficult getting the data right than solving the technical problem.

As a decision oriented person you don't need to know the technical methods, any more than you need database expertise to create reports.  This books aims at the business problem and solutions, with a strong numerical focus.  Usually with basic math.   The title of the book NumberSense, is that quality of understanding when an analysis is right or going wrong, and what to do about that.   The data in an analysis does not have to be BIG or even complex, just correctly addressed.   The book and more about it:

NumberSense: How to Use Big Data to Your Advantage    by Kaiser Fung .

See also his Numbers Rule Your World blog site for day to day examples.

Examples covered: 

" ... How does the college ranking system really work?
Can an obesity measure solve America's biggest healthcare crisis?
Should you trust current unemployment data issued by the government?
How do you improve your fantasy sports team?
Should you worry about businesses that track your data?

Don't take for granted statements made in the media, by our leaders, or even by your best friend. We're on information overload today, and there's a lot of bad information out there.
Numbersense gives you the insight into how Big Data interpretation works--and how it too often doesn't work. You won't come away with the skills of a professional statistician. But you will have a keen understanding of the data traps even the best statisticians can fall into, and you'll trust the mental alarm that goes off in your head when something just doesn't seem to add up.... "

Do Data Scientists Scale?

Andrew Brust on the nature of data scientists and the scalability of what they do, or claim to do.  I agree with many of the points made.   I will summarize my view .. the term Data Scientist implies that there is some deep science involved, and simplicity cannot be part of that.  It makes things harder than they need to be.  Analytics should only be as hard as it needs to be.  Read the whole article.

The Nature of the Data Scientist

Stephen Few writes  about the nature of the term 'data scientist'.  I personally don't like the term,  I prefer the term 'analyst', which has fewer implications of complexity. The term also is more of a grouping of technical and business interests meant to solve problems.  Needing a 'scientist' may be overdoing it.  Few does a good job of addressing the nature of the term.