Stock Prognosticators? (6/11/14)
A first of its kind study by researchers at the University of Iowa suggests Yahoo’s finance message boards have a small degree of ability to predict stock price movements.
The study, “Stock Chatter: Using stock sentiment to predict price direction,” also found that more than two-thirds of the message board comments had nothing to do with finance.
The researchers analyzed 70,000 posts by more than 7,000 commenters on Yahoo’s finance message boards from April to June 2011. They determined what sentiment, if any, they expressed about 11 Fortune 500 stocks, either bullish, bearish, or neutral. The researchers then looked at the movement of those stocks’ prices the next day. Depending on the model the researchers used to classify the statements, they found that the sentiment expressed on the message boards accurately reflected the price movement anywhere between 52 and 64 percent of the time.
Michael Rechenthin, who conducted the study as a doctoral student in the Tippie College of Business, says that while the lower accuracy figures can be attributed to randomness, the 64 percent figure is statistically significant and shows a small degree of predictive ability.
The study found the predictive ability lasted only one day, though, and disappeared on follow-up days.
The study also found that only a small number of users produced the largest number of comments, and those comments seem to be responsible for whatever predictive quality the message boards had. Rechenthin says only 3 percent of the commenters wrote 50 percent of the posts during the study period, and 11 percent wrote 75 percent of the posts.
Those extreme users also tended to be the most opinionated, he says, and when they were removed from the study, what predictive ability the boards had all but disappeared. He says the study did not determine whether this was because the most forceful opinions affected stock price, or if those with stronger opinions were more seasoned market observers and able to see trends.
The study also found that, as one might expect of public message boards, many of the posts were completely irrelevant to the topic. They determined that 68 percent of posts were things like advertisements, spam, flame wars, or comments by trolls.
Politics tended to be the most common off-topic topics, he says, with President Obama and former President Bush frequent topics of criticism. The study period also included former Rep. Anthony Weiner’s resignation from Congress after texting inappropriate photos, and many comments were related to him.
Rechenthin cautions that as this is the first study of its type and follow-up studies will be needed using larger data sets. The research was published recently in the journal Algorithmic Finance and was coauthored with W. Nick Street, professor of management sciences in the Tippie College of Business, and Padmini Srinivasan, professor of computer science at the University of Iowa.
Drafty Research (5/6/14)
When general managers gather in New York for the NFL draft this week, they’ll be awash in statistics, scouting reports, interview data, and video clips as they look for a way to digest it all and make the best draft selections for their respective teams.
How should the selection be made? They can use a rule of thumb, like filling the biggest hole in their roster, or picking the best player available. But how is best measured? University of Iowa professor Jeff Ohlmann says several teams are arming their staffs with information based on analytics in efforts to gain any edge that they can.
One of Ohlmann’s research focuses is how a sports team can optimize its draft selections, and this work has led to the development of a smartphone app to help fantasy football and baseball team owners pick their teams. Ohlmann says that a sports draft is a great example of what is called a “sequential decision-making problem with uncertainty,” an area of research that attracts interest of both academics and practitioners. For example, logistics companies need to design routes to supply goods without knowing exactly how much product will be required by the retailer or precisely how long it may take to get there.
In football, the uncertainty lies in not knowing how well a player may perform in the NFL, or even if that player will be available when a team is “on the clock” because some other team may pick him first. Using probability distributions to model the uncertainty, Ohlmann and other researchers have developed models that use mathematical techniques to maximize the expected value of the players which a team drafts. At its heart, the model tries to help general managers overcome the fundamental handicap of not knowing what players will be available to draft in future rounds.
In Ohlmann’s approach, a team projects opposing teams’ selections to forecast which players will be available to the team in future rounds. He says that while there will surely be errors in guessing which players other teams will select, the errors often cancel each other out to create a sufficiently accurate forecast of the draft as a whole. The result, he says, is a model that produces a draft strategy that typically dominates alternative drafting rules-of-thumb.
“Rules-of-thumb have flaws in them and you can do better if you try to predict what will happen in the future,” he says. “A draft strategy based on analytics isn’t guaranteed to dominate, but more often than not, it will be the best strategy.”
He acknowledges data analysis is not a crystal ball and will not be 100 percent accurate in identifying which players are going to be successful. “But it may help avoid a bad decision in cases when the front office personnel are 'fooled by their eyes’ and let emotions affect their decisions.”
Ohlmann, who teaches in the department of Management Sciences in the Tippie College of Business, has used his sports research to develop and teach a first-year seminar, Sports Analytics, to introduce analytical tools to students using a topic that many students are naturally interested in.
“My goal is to show students how to formulate sports-related questions and then use data and math to try to answer them rather than just qualitatively debating them,” he says.
Is Big Data Dating the Key to Long-Lasting Romance? (BBC News, 3/24/14)
If you want to know if a prospective date is relationship material, just ask them three questions, says Christian Rudder, one of the founders of U.S. Internet dating site OKCupid.
- "Do you like horror movies?"
- "Have you ever traveled around another country alone?"
- "Wouldn't it be fun to chuck it all and go live on a sailboat?"
Why? Because these are the questions first date couples agree on most often, he says.
Mr. Rudder discovered this by analysing large amounts of data on OKCupid members who ended up in relationships.
Dating agencies like OKCupid, Match.com—which acquired OKCupid in 2011 for $50m (£30m)—eHarmony and many others, amass this data by making users answer questions about themselves when they sign up.
Some agencies ask as many as 400 questions, and the answers are fed in to large data repositories. Match.com estimates that it has more than 70 terabytes (70,000 gigabytes) of data about its customers.
Applying big data analytics to these treasure troves of information is helping the agencies provide better matches for their customers. And more satisfied customers mean bigger profits.
U.S. Internet dating revenues top $2bn (£1.2bn) annually, according to research company IBISWorld. Just under one in 10 of all American adults have tried it.
The market for dating using mobile apps is particularly strong and is predicted to grow from about $1bn in 2011 to $2.3bn by 2016, according to Juniper Research.
There is, however, a problem: people lie.
To present themselves in what they believe to be a better light, the information customers provide about themselves is not always completely accurate: men are most commonly economical with the truth about age, height, and income, while with women it's age, weight, and build.
Mr. Rudder adds that many users also supply other inaccurate information about themselves unintentionally.
"My intuition is that most of what users enter is true, but people do misunderstand themselves," he says.
For example, a user may honestly believe that they listen mostly to classical music, but analysis of their iTunes listening history or their Spotify playlists might provide a far more accurate picture of their listening habits.
Inaccurate data is a problem because it can lead to unsuitable matches, so some dating agencies are exploring ways to supplement user-provided data with that gathered from other sources.
With users' permission, dating services could access vast amounts of data from sources including their browser and search histories, film-viewing habits from services such as Netflix and Lovefilm, and purchase histories from online shops like Amazon.
But the problem with this approach is that there is a limit to how much data is really useful, Mr. Rudder believes.
"We've found that the answers to some questions provide useful information, but if you just collect more data you don't get high returns on it," he says.
This hasn't stopped Hinge, a Washington, D.C.-based dating company, gathering information about its customers from their Facebook pages.
The data is likely to be accurate because other Facebook users police it, Justin McLeod, the company's founder, believes.
"You can't lie about where you were educated because one of your friends is likely to say, 'You never went to that school'," he points out.
It also infers information about people by looking at their friends, Mr. McLeod says.
"There is definitely useful information contained in the fact that you are a friend of someone."
Hinge suggests matches with people known to their Facebook friends.
"If you show a preference for people who work in finance, or you tend to like Bob's friends but not Ann's, we use that when we curate possible matches," he explains.
The pool of potential matches can be considerable, because Hinge users have an average of 700 Facebook friends, Mr McLeod adds.
But it turns out that algorithms can produce good matches without asking users for any data about themselves at all.
For example, Dr. Kang Zhao, an assistant professor at the University of Iowa and an expert in business analytics and social network analysis, has created a match-making system based on a technique known as collaborative filtering.
Dr. Zhao's system looks at users' behaviour as they browse a dating site for prospective partners, and at the responses they receive from people they contact.
"If you are a boy we identify people who like the same girls as you—which indicates similar taste—and people who get the same response from these girls as you do—which indicates similar attractiveness," he explains.
Dr. Zhao's algorithm can then suggest potential partners in the same way websites like Amazon or Netflix recommend products or movies, based on the behaviour of other customers who have bought the same products, or enjoyed the same films.
Internet dating may be big business, but no one has yet devised the perfect matching system. It may well be that the secret of true love is simply not susceptible to big data or any other type of analysis.
"Two people may have exactly the same iTunes history," OKCupid's Christian Rudder concludes, "but if one doesn't like the other's clothes or the way they look then there simply won't be any future in that relationship."
- Dating Algorithm Makes Matches (KGAN, 2/14/14)
- Machine Learning + Love (WNYC, 2/12/14)
- UI Developed Algorithm Gives Online Dating a New Spin (KCRG, 2/5/14)
- Predicting Stock Prices (1/31/14)
- UI Team's Dating Algorithm Could Mean Better Matches (Iowa City Press-Citizen, 1/5/14)
- Can a New Online Dating Algorithm Transform Your Love Life? (Refinery 29, 12/11/13)
- Need Love? There's a New Algorithm for That (Jezebel, 12/11/13)
- University of Iowa Professor Revamps Online Dating with New Algorithm (Philly.com, 12/11/13)
- A New Online Dating Algorithm Will Match You With Someone You Might Actually Have A Chance With (Business Insider, 12/11/13)
- Researchers: Dating Sites Have It All Wrong (Consumer Affairs, 12/9/13)
- Why the Future of Online Dating Relies on Ignoring You (Forbes, 12/7/13)
- UI Researchers: Netflix-Style Tracking Can Increase Online Dating Success (Des Moines Register, 12/4/13)
- Love Connection (12/4/13)
- University of Iowa Professor Advises on Disaster Relief Logistics (The Gazette, 11/16/13)
- Getting Aid to Philippines Is Challenging, Costly (KCRG, 11/13/13)
- Mathematics Turned to Helping Disaster Relief After Philippine Typhoon (UPI, 11/12/13)
- Thomas Elected to INFORMS Transportation Science and Logistics Society (10/29/13)
- Pant Named Associate Editor of the Year (10/11/13)
- Discover 11 Hot College Majors That Lead to Jobs (U.S. News & World Report, 9/10/13)
- More Shipments a Good Sign for the Economy (The Gazette, 8/24/13)
- University of Iowa Offers New Business Analytics Major for Undergrads (Data Informed, 8/8/13)
- Mining Big Data (The Press-Citizen, 8/3/13)
- New Safety Regulations Limit Hours for Truck Drivers (KWWL, 7/2/13)
- Tippie Faculty Member Releases Text on Analyzing Big Data (5/20/13)
- BTA Students Attend AITP-NCC (4/8/13)