In a couple of days, theWall Street Journalwill come out with its November survey of economists’ forecasts. It’s a particularly sensitive time, with elections in a few days and President Trump attacking the Federal Reserve for for raising interest rates. It’s a good time to recall major forecasting gaffes of the past. In 1987, best-sellingContinue reading “Examples of Bad Forecasting”
Category Archives: Analytics
100 years of variance
It is 100 years since R A Fischer introduced the concept of “variance“(in his 1918 paper “The Correlation Between Relatives on the Supposition of Mendelian Inheritance“). There is much that statistics has given us in the century that followed. Randomized clinical trials, and the means to analyze them, moved medicine fully into the modern, science-based era.Continue reading “100 years of variance”
Course Spotlight: Structural Equation Modelling (SEM)
SEM stands for “structural equation modeling,” and we are fortunate to have Prof. Randall Schumacker teaching this subject at Statistics.com. Randy created the Structural Equation Modeling (SEM) journal in 1994 and the Structural Equation Modeling Special Interest Group (SIG) at the American Educational Research Association (AERA) He has also co-authored several books, including: A Beginner’sContinue reading “Course Spotlight: Structural Equation Modelling (SEM)”
Benford’s Law Applies to Online Social Networks
Fake social media accounts and Russian meddling in US elections have been in the news lately, with Mark Zuckerberg (Facebook founder) testifying this week before the US Congress. Dr. Jen Golbeck, who teaches Network Analysis at Statistics.com, published an ingenious way to determine whether a Facebook, Twitter or other social media account is fraudulent. HerContinue reading “Benford’s Law Applies to Online Social Networks”
Masters Programs versus an Online Certificate in Data Science from Statistics.com
We just attended the analytics conference of INFORMS’ (The Institute for Operations Research and the Management Sciences) this week in Baltimore, and they held a special meeting for directors of academic analytics programs to better align what universities are producing with what industry is seeking. The number of such programs is still growing rapidly (>200),Continue reading “Masters Programs versus an Online Certificate in Data Science from Statistics.com”
Course Spotlight: Predictive Analytics
Predicting whether an internet user will click on a link or buy a product, whether an insurance claim is fraudulent, whether a home mortgage will be paid on time (or early), how much a house will sell for, what internet ad you should see next, whether a discharged patient will need to return to theContinue reading “Course Spotlight: Predictive Analytics”
“Money and Brains” and “Furs and Station Wagons”
“Money and Brains” and “Furs and Station Wagons” were evocative customer shorthands that the marketing company Claritas came up with over a half century ago. These names, which facilitated the work of marketers and sales people, were shorthand descriptions of segments of customers identified through statistical cluster analysis. Cluster analysis is also used in marketContinue reading ““Money and Brains” and “Furs and Station Wagons””
Kidney Donations – Optimizing Them
Did you know that there is a country in which kidneys can be legally bought and sold? That country is Iran, and there are actually two models for payment. In one part of the country, donors receive a flat fee, and in the other part of the country prices are negotiated. This point wasContinue reading “Kidney Donations – Optimizing Them”
Course Spotlight: The Text Analytics Sequence
Text analytics or text mining is the natural extension of predictive analytics, and Statistics.com’s text analytics program starts Feb. 6. Text analytics is now ubiquitous and yields insight in: Marketing: Voice of the customer, social media analysis, churn analysis, market research, survey analysis Business: Competitive intelligence, document categorization, human resources (voice of the employee), recordsContinue reading “Course Spotlight: The Text Analytics Sequence”
Course Spotlight: Constrained Optimization
Say you operate a tank farm (to store and sell fuel). How much of each fuel grade should you buy? You have specified flow and storage capacities, constraints on what types of fuels can be stored in which tanks, prior contractual obligations about minimum monthly deliveries and incoming supplies, plus the opportunity to sell onContinue reading “Course Spotlight: Constrained Optimization”
Big Data and Clinical Trials in Medicine
There was an interesting article a couple of weeks ago in the New York Times magazine section on the role that Big Data can play in treating patients — discovering things that clinical trials are too slow, too expensive, and too blunt to find. The story was about a very particular set of lupus symptoms,Continue reading “Big Data and Clinical Trials in Medicine”
Industry Spotlight: The brand premium for Chanel and Harvard
The classic illustration of the power of brand is perfume – expensive perfumes may cost just a few dollars to produce but can be sold for more than $500 due to the cachet afforded by the brand. David Malan’s Computer Science course at Harvard, CSCI E-50, provides an interesting parallel in the education world. It’sContinue reading “Industry Spotlight: The brand premium for Chanel and Harvard”
Convoys
Ever wonder why, in World War II, ships in convoys were safer than ships traveling on their own? Most people assume it was due to the protection afforded by military escort vessels, of which there was a limited supply (insufficient to protect ships traveling on their own). Actually, most of the benefit came from theContinue reading “Convoys”
Dialects
When talking to several people, do you address them as “you guys”? “Y’all”? Just “you”? And is the carbonated soft drink “soda” or “pop?” Maps based on survey responses to questions like this were published in the Harvard Dialect Survey in 2003. Josh Katz took the data and produced extended visualizations and, last month, aContinue reading “Dialects”
Needle in a Haystack
What’s the probability that the NSA examined the metadata for your phone number in 2013? According to John Inglis, Deputy Director at the NSA, it’s about 0.00001, or 1 in 100,000. A surprisingly small number, given what we’ve all been reading in the media about NSA’s massive data collection effort. Of course, that’s an unconditionalContinue reading “Needle in a Haystack”
Predictive Modeling and Typhoon Relief
The devastation wrought by Super-Typhoon Haiyan in the Philippines is the biggest test yet for the nascent technology of “artificial intelligence disaster response,” a phrase used by Patrick Meier, a pioneer in the field. When disaster strikes, a flood of social media posts and tweets ensues. There is useful information in the data flood, butContinue reading “Predictive Modeling and Typhoon Relief”
Terrorist Clusters
The “righteous vengeance gun attack” is just one of 10 types of terrorism identified by Chenoweth and Lowham via statistical clustering techniques. Another cluster is “bombings of a public population where a liberation group takes responsibility.” You can read about the 10 clusters, and the 44 dichotomous variables (suicide or not, bombing or not, religiousContinue reading “Terrorist Clusters”
Statistics.com Partners With CrowdANALYTIX to Offer New Online Course With Crowdsource Contest As Project
Crowdsourcing, using the power of the crowd to solve problems, has been used for many functions and tasks, including predictive modeling (like the 2009 Netflix Contest). Typically, problems are broadcast to an unknown group of statistical modelers on the Internet, and solutions are sought. Every crowdsourced project harnesses the power of the community to findContinue reading “Statistics.com Partners With CrowdANALYTIX to Offer New Online Course With Crowdsource Contest As Project”
Illuminate, Iterate, Involve, Involvement, Iteration, Insight
I did not start off in the field of statistics; my first real job was as a diplomat. And from my undergraduate days I recall a professor who taught a cultural history of Russia. He was one of the world’s top experts. Possessed of a tremendous store of knowledge (a leading author in the field,Continue reading “Illuminate, Iterate, Involve, Involvement, Iteration, Insight”